[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:40 dist-test-slave-0kls krb5kdc[4069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:40 dist-test-slave-0kls krb5kdc[4069](info): set up 2 sockets
Aug 06 06:50:40 dist-test-slave-0kls krb5kdc[4069](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 06:50:41.421739 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:41.541239 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39303
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:41.621924 4082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:41.622159 4082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:41.622196 4082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:41.623377 4082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:41.623440 4082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:41.623466 4082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:41.623499 4082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:41.623523 4082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:41.624980 4082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:41.625250 4082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:41.625489 4082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:41.630640 4087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:41.630640 4090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:41.630774 4088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:41.631021 4082 server_base.cc:1034] running on GCE node
I20260806 06:50:41.631258 4082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:41.631546 4082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:41.632692 4082 hybrid_clock.cc:648] HybridClock initialized: now 1785999041632676 us; error 30 us; skew 500 ppm
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:41.635296 4082 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:41.635895 4082 webserver.cc:460] Webserver started at http://127.3.247.190:38293/ using document root <none> and password file <none>
I20260806 06:50:41.636138 4082 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:41.636183 4082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:41.636276 4082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:41.637324 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "a671089aca804f0f95c7be9e5817ba13"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "ab8dd0d566cb3541f8cd342a40fd2d0f"
server_key_iv: "b3192ac06d60730b5d9fac0927c4691a"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.637641 4082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "a671089aca804f0f95c7be9e5817ba13"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "ab8dd0d566cb3541f8cd342a40fd2d0f"
server_key_iv: "b3192ac06d60730b5d9fac0927c4691a"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.639045 4082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:41.639768 4097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:41.639990 4082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:41.640061 4082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "a671089aca804f0f95c7be9e5817ba13"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "ab8dd0d566cb3541f8cd342a40fd2d0f"
server_key_iv: "b3192ac06d60730b5d9fac0927c4691a"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.640115 4082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:41.658495 4082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:41.660523 4082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:41.660638 4082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:41.664625 4082 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39303
I20260806 06:50:41.664680 4149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39303 every 8 connection(s)
I20260806 06:50:41.665086 4082 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:41.666165 4150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:41.666301 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4082
I20260806 06:50:41.666393 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:41.667168 4062 external_mini_cluster.cc:1428] Setting key 81a7faff4ce11f6bd2e71e006ad70725
I20260806 06:50:41.669167 4150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13: Bootstrap starting.
I20260806 06:50:41.669730 4150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:41.670045 4150 log.cc:824] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:41.670846 4150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13: No bootstrap required, opened a new log
I20260806 06:50:41.672812 4150 raft_consensus.cc:348] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } }
I20260806 06:50:41.673008 4150 raft_consensus.cc:374] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:41.673062 4150 raft_consensus.cc:729] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a671089aca804f0f95c7be9e5817ba13, State: Initialized, Role: FOLLOWER
I20260806 06:50:41.673244 4150 consensus_queue.cc:260] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } }
I20260806 06:50:41.673329 4150 raft_consensus.cc:388] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:41.673384 4150 raft_consensus.cc:482] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:41.673437 4150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:41.674113 4150 raft_consensus.cc:504] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } }
I20260806 06:50:41.674249 4150 leader_election.cc:302] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a671089aca804f0f95c7be9e5817ba13; no voters:
I20260806 06:50:41.674432 4150 leader_election.cc:288] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:41.674456 4155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:41.674571 4155 raft_consensus.cc:686] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [term 1 LEADER]: Becoming Leader. State: Replica: a671089aca804f0f95c7be9e5817ba13, State: Running, Role: LEADER
I20260806 06:50:41.674696 4150 sys_catalog.cc:564] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:41.674726 4155 consensus_queue.cc:237] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } }
I20260806 06:50:41.675143 4155 sys_catalog.cc:455] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a671089aca804f0f95c7be9e5817ba13. Latest consensus state: current_term: 1 leader_uuid: "a671089aca804f0f95c7be9e5817ba13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } } }
I20260806 06:50:41.675401 4155 sys_catalog.cc:458] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:41.675328 4156 sys_catalog.cc:455] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a671089aca804f0f95c7be9e5817ba13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a671089aca804f0f95c7be9e5817ba13" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39303 } } }
I20260806 06:50:41.675815 4156 sys_catalog.cc:458] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:41.676010 4164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:41.676513 4164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:41.678272 4164 catalog_manager.cc:1347] Generated new cluster ID: 44867744983548f7aac31eb16ffe35c3
I20260806 06:50:41.678324 4164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:41.683670 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:41.668087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.3.247.190:39303)
0806 06:50:41.668263 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:41.668273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:41.668479 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:50:41.670658 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:41.670666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:41.670724 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:41.670883 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:41.672311 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.673856 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.675273 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.675900 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.676009 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:41.679931 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:41.679935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:41.679936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:41.679983 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:41.681695 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:41.682199 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:41.682202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:41.682203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:41.682311 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:41.682580 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:41.682581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:41.682582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:41.682831 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:41.683047 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:41.683291 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 06:50:41.683438 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260806 06:50:41.702311 4164 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:41.703106 4164 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:41.707165 4164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a671089aca804f0f95c7be9e5817ba13: Generated new TSK 0
I20260806 06:50:41.707396 4164 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:41.740466 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:41.830225 4176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:41.830456 4176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:41.830494 4176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:41.831624 4176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:41.831686 4176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:41.831732 4176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:41.833186 4176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:41.833448 4176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:41.833664 4176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:41.838510 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:41.838517 4184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:41.838517 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:41.838869 4176 server_base.cc:1034] running on GCE node
I20260806 06:50:41.839061 4176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:41.839300 4176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:41.840435 4176 hybrid_clock.cc:648] HybridClock initialized: now 1785999041840418 us; error 31 us; skew 500 ppm
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:41.843060 4176 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:41.843441 4176 webserver.cc:460] Webserver started at http://127.3.247.129:43483/ using document root <none> and password file <none>
I20260806 06:50:41.843618 4176 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:41.843657 4176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:41.843781 4176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:41.844792 4176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "e0b3672b1109490aa2d267bd8402c1e0"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "991477412be0549f45e4b82dbae489a9"
server_key_iv: "a5ae77b216b8f2041f2e8d7c8d7082bb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.845103 4176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "e0b3672b1109490aa2d267bd8402c1e0"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "991477412be0549f45e4b82dbae489a9"
server_key_iv: "a5ae77b216b8f2041f2e8d7c8d7082bb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.846392 4176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:41.847098 4191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:41.847324 4176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:41.847395 4176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "e0b3672b1109490aa2d267bd8402c1e0"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "991477412be0549f45e4b82dbae489a9"
server_key_iv: "a5ae77b216b8f2041f2e8d7c8d7082bb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.847469 4176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:41.856459 4176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:41.858999 4176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:41.859153 4176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:41.859390 4176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:41.859704 4176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:41.859755 4176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:41.859797 4176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:41.859815 4176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:41.865844 4176 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:45637
I20260806 06:50:41.865891 4304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:45637 every 8 connection(s)
I20260806 06:50:41.866253 4176 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:41.875581 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:41.866991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:50509 (local address 127.3.247.190:39303)
0806 06:50:41.867056 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:41.867059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:41.867280 (+ 221us) server_negotiation.cc:407] Connection header received
0806 06:50:41.867822 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:41.867823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:41.867832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:41.867890 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:41.868897 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.869395 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.870410 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.870611 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.870667 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:41.872765 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:41.872767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:41.872767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:41.872773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:41.874233 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:41.874663 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:41.874664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:41.874665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:41.874723 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:41.874953 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:41.874954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:41.874954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:41.875054 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:41.875103 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:41.875403 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 06:50:41.875484 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:50:41.875798 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:41.866963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39303 (local address 127.3.247.129:50509)
0806 06:50:41.867151 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:41.867194 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:41.867686 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:41.867994 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:41.867997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:41.868585 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:41.868809 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:41.868813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.869502 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.869503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:41.870312 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:41.870314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:41.870686 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:41.870687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:41.870789 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:41.871272 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:41.871280 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:41.872660 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:41.874309 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:41.874316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:41.874318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:41.874587 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:41.874781 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:41.874781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:41.874782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:41.874899 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:41.875108 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:41.875112 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:41.875275 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 06:50:41.875359 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:50:41.875543 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260806 06:50:41.876180 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4176
I20260806 06:50:41.876281 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:41.876343 4305 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39303
I20260806 06:50:41.876441 4305 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:41.876451 4062 external_mini_cluster.cc:1428] Setting key b33e5d6b01ca7eb56fce920790cea383
I20260806 06:50:41.876631 4305 heartbeater.cc:511] Master 127.3.247.190:39303 requested a full tablet report, sending...
I20260806 06:50:41.877205 4113 ts_manager.cc:194] Registered new tserver with Master: e0b3672b1109490aa2d267bd8402c1e0 (127.3.247.129:45637)
I20260806 06:50:41.877914 4113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:50509
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:41.900919 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:41.982645 4310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:41.982846 4310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:41.982870 4310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:41.983973 4310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:41.984010 4310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:41.984061 4310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:41.985517 4310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:41.985697 4310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:41.985935 4310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:41.990823 4318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:41.990967 4316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:41.990823 4310 server_base.cc:1034] running on GCE node
W20260806 06:50:41.990924 4315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:41.991219 4310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:41.991467 4310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:41.992630 4310 hybrid_clock.cc:648] HybridClock initialized: now 1785999041992626 us; error 39 us; skew 500 ppm
Aug 06 06:50:41 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:41.995162 4310 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:41.995597 4310 webserver.cc:460] Webserver started at http://127.3.247.130:35099/ using document root <none> and password file <none>
I20260806 06:50:41.995834 4310 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:41.995903 4310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:41.996064 4310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:41.997109 4310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "51f72c97127e4945bd08952b86d43415"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "9e6d7a28382d25be293c71335faaaa2e"
server_key_iv: "df2a938f7260fe20cfdf25662fb6e0fb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.997480 4310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "51f72c97127e4945bd08952b86d43415"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "9e6d7a28382d25be293c71335faaaa2e"
server_key_iv: "df2a938f7260fe20cfdf25662fb6e0fb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.998865 4310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:41.999576 4325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:41.999744 4310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:41.999845 4310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "51f72c97127e4945bd08952b86d43415"
format_stamp: "Formatted at 2026-08-06 06:50:41 on dist-test-slave-0kls"
server_key: "9e6d7a28382d25be293c71335faaaa2e"
server_key_iv: "df2a938f7260fe20cfdf25662fb6e0fb"
server_key_version: "encryptionkey@0"
I20260806 06:50:41.999924 4310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:42.019302 4310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:42.021271 4310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:42.021450 4310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:42.021692 4310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:42.022042 4310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:42.022099 4310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.022141 4310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:42.022168 4310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.028164 4310 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:33389
I20260806 06:50:42.028209 4438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:33389 every 8 connection(s)
I20260806 06:50:42.028549 4310 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:42.035712 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4310
I20260806 06:50:42.035811 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:42.035977 4062 external_mini_cluster.cc:1428] Setting key b447500212070f9403165b1975808004
I20260806 06:50:42.038273 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.029345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:45047 (local address 127.3.247.190:39303)
0806 06:50:42.029420 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:42.029422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:42.029644 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:50:42.030194 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:42.030195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:42.030205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:42.030266 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:42.031211 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.031699 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.032674 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.032875 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.032930 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.035052 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:42.035053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:42.035054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:42.035061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:42.036715 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.037205 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.037206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.037206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.037266 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.037497 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.037497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.037498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.037638 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:42.037704 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:42.038032 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 06:50:42.038136 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:50:42.038249 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.029334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39303 (local address 127.3.247.130:45047)
0806 06:50:42.029517 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:42.029555 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:42.030107 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:42.030339 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:42.030343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:42.030931 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:42.031123 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:42.031128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.031783 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.031783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:42.032594 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:42.032595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.032952 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.032953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:42.033053 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.033516 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:42.033523 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:42.034956 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:42.036872 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:42.036879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:42.036881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:42.037131 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:42.037334 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:42.037335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:42.037335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:42.037435 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:42.037700 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:42.037704 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:42.037867 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 06:50:42.037967 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:50:42.038086 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260806 06:50:42.038897 4439 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39303
I20260806 06:50:42.038988 4439 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:42.039177 4439 heartbeater.cc:511] Master 127.3.247.190:39303 requested a full tablet report, sending...
I20260806 06:50:42.039556 4113 ts_manager.cc:194] Registered new tserver with Master: 51f72c97127e4945bd08952b86d43415 (127.3.247.130:33389)
I20260806 06:50:42.039932 4113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:45047
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:42.062516 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:42.145887 4444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:42.146178 4444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:42.146220 4444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:42.147481 4444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:42.147559 4444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:42.147606 4444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:42.149150 4444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39303
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:42.149433 4444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:42.149675 4444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:42.154706 4450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:42.154817 4452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:42.154858 4449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:42.154924 4444 server_base.cc:1034] running on GCE node
I20260806 06:50:42.155145 4444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:42.155375 4444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:42.156545 4444 hybrid_clock.cc:648] HybridClock initialized: now 1785999042156531 us; error 22 us; skew 500 ppm
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:42.159225 4444 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:42.159713 4444 webserver.cc:460] Webserver started at http://127.3.247.131:33127/ using document root <none> and password file <none>
I20260806 06:50:42.159947 4444 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:42.160014 4444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:42.160173 4444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:42.161278 4444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "e697fb1280e64ac587f77b6535f0e19b"
format_stamp: "Formatted at 2026-08-06 06:50:42 on dist-test-slave-0kls"
server_key: "67e67aad891600b5ce79505855adcdde"
server_key_iv: "8e83fe5a60736c79db074179cd4494ed"
server_key_version: "encryptionkey@0"
I20260806 06:50:42.161679 4444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "e697fb1280e64ac587f77b6535f0e19b"
format_stamp: "Formatted at 2026-08-06 06:50:42 on dist-test-slave-0kls"
server_key: "67e67aad891600b5ce79505855adcdde"
server_key_iv: "8e83fe5a60736c79db074179cd4494ed"
server_key_version: "encryptionkey@0"
I20260806 06:50:42.163026 4444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:42.163766 4459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.163956 4444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.164062 4444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "e697fb1280e64ac587f77b6535f0e19b"
format_stamp: "Formatted at 2026-08-06 06:50:42 on dist-test-slave-0kls"
server_key: "67e67aad891600b5ce79505855adcdde"
server_key_iv: "8e83fe5a60736c79db074179cd4494ed"
server_key_version: "encryptionkey@0"
I20260806 06:50:42.164134 4444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:42.192250 4444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:42.194638 4444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:42.194795 4444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:42.195031 4444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:42.195335 4444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:42.195410 4444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.195452 4444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:42.195480 4444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:42.201233 4444 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:43115
I20260806 06:50:42.201275 4572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:43115 every 8 connection(s)
I20260806 06:50:42.201586 4444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:42.207711 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4444
I20260806 06:50:42.207772 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:42.207921 4062 external_mini_cluster.cc:1428] Setting key 4dcc5087a33c2a9fe4537a727f87e7f4
I20260806 06:50:42.210696 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.202323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:42651 (local address 127.3.247.190:39303)
0806 06:50:42.202364 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:42.202366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:42.202534 (+ 168us) server_negotiation.cc:407] Connection header received
0806 06:50:42.203012 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:42.203013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:42.203022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:42.203081 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:42.204093 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.204579 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.205571 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.205744 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.205839 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.208117 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:42.208118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:42.208119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:42.208126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:42.209597 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.210007 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.210008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.210008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.210053 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.210212 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.210213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.210213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.210300 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:42.210344 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:42.210580 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 06:50:42.210637 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:42.210917 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.202270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39303 (local address 127.3.247.131:42651)
0806 06:50:42.202424 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:42.202464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:42.202927 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:42.203134 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:42.203138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:42.203737 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:42.203987 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:42.203993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.204702 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.204703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:42.205499 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:42.205501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.205878 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.205879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:42.205979 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.206480 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:42.206488 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:42.208023 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:42.209665 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:42.209670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:42.209672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:42.209949 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:42.210092 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:42.210093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:42.210093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:42.210179 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:42.210349 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:42.210353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:42.210469 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 06:50:42.210553 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:50:42.210666 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260806 06:50:42.211421 4573 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39303
I20260806 06:50:42.211527 4573 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:42.211700 4573 heartbeater.cc:511] Master 127.3.247.190:39303 requested a full tablet report, sending...
I20260806 06:50:42.212049 4113 ts_manager.cc:194] Registered new tserver with Master: e697fb1280e64ac587f77b6535f0e19b (127.3.247.131:43115)
I20260806 06:50:42.212417 4113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:42651
I20260806 06:50:42.220193 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:42.243647 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.235290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.3.247.129:45637)
0806 06:50:42.235504 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:42.235508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:42.235519 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:42.235548 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:42.235550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:42.235565 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:42.235634 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:42.236199 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.236957 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.237666 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.238077 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.238136 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.240705 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:42.240717 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:42.240718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:42.240726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:42.242346 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.242831 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.242835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.242835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.242910 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.243095 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.243096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.243096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.243244 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:42.243354 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:42.243459 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:50:42.243527 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
W20260806 06:50:42.244120 4219 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43610
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:42.268618 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:42.261095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.3.247.129:45637)
0806 06:50:42.261172 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:42.261175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:42.261267 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:50:42.261368 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:42.261369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:42.261391 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:42.261460 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:42.262140 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.262641 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.263440 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:42.263658 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:42.263739 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:42.266108 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:42.266111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:42.266111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:42.266122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:42.267486 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.267932 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.267933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.267933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.267997 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:42.268190 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:42.268190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:42.268191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:42.268291 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:42.268395 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:42.268478 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:50:42.268546 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:50:42.269215 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4176
I20260806 06:50:42.273700 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4310
I20260806 06:50:42.278247 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4444
I20260806 06:50:42.283321 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4082
2026-08-06T06:50:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1923 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4587](info): set up 2 sockets
Aug 06 06:50:42 dist-test-slave-0kls krb5kdc[4587](info): commencing operation
krb5kdc: starting...
W20260806 06:50:43.349603 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:43.488962 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39795
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:43.573237 4600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:43.573508 4600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:43.573549 4600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:43.574803 4600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:43.574879 4600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:43.574903 4600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:43.574925 4600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:43.574954 4600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:43.576472 4600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:43.576767 4600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:43.577010 4600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:43.582163 4600 server_base.cc:1034] running on GCE node
W20260806 06:50:43.582109 4605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.582118 4606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.582116 4608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:43.582633 4600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:43.582945 4600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:43.584103 4600 hybrid_clock.cc:648] HybridClock initialized: now 1785999043584083 us; error 41 us; skew 500 ppm
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:43.586889 4600 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:43.587358 4600 webserver.cc:460] Webserver started at http://127.3.247.190:33187/ using document root <none> and password file <none>
I20260806 06:50:43.587616 4600 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:43.587689 4600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:43.587852 4600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:43.588956 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "e60521a5b6fc49e5bc419126acdbdb46"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "e7f2241815b601b01e03e7cf9b789ff2"
server_key_iv: "ba1cc6e4827e06509d4b91d1c786499b"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.589341 4600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "e60521a5b6fc49e5bc419126acdbdb46"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "e7f2241815b601b01e03e7cf9b789ff2"
server_key_iv: "ba1cc6e4827e06509d4b91d1c786499b"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.590849 4600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:43.591614 4615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.591810 4600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:43.591907 4600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "e60521a5b6fc49e5bc419126acdbdb46"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "e7f2241815b601b01e03e7cf9b789ff2"
server_key_iv: "ba1cc6e4827e06509d4b91d1c786499b"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.591991 4600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:43.607950 4600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:43.610952 4600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:43.611138 4600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:43.615154 4600 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39795
I20260806 06:50:43.615175 4667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39795 every 8 connection(s)
I20260806 06:50:43.615561 4600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:43.616534 4668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:43.618957 4668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46: Bootstrap starting.
I20260806 06:50:43.619527 4668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:43.619762 4668 log.cc:824] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:43.620477 4668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46: No bootstrap required, opened a new log
I20260806 06:50:43.622174 4668 raft_consensus.cc:348] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } }
I20260806 06:50:43.622331 4668 raft_consensus.cc:374] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:43.622355 4668 raft_consensus.cc:729] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60521a5b6fc49e5bc419126acdbdb46, State: Initialized, Role: FOLLOWER
I20260806 06:50:43.622501 4668 consensus_queue.cc:260] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } }
I20260806 06:50:43.622582 4668 raft_consensus.cc:388] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:43.622607 4668 raft_consensus.cc:482] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:43.622638 4668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:43.623145 4668 raft_consensus.cc:504] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } }
I20260806 06:50:43.623231 4668 leader_election.cc:302] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e60521a5b6fc49e5bc419126acdbdb46; no voters:
I20260806 06:50:43.623381 4668 leader_election.cc:288] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:43.623469 4671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:43.623656 4671 raft_consensus.cc:686] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [term 1 LEADER]: Becoming Leader. State: Replica: e60521a5b6fc49e5bc419126acdbdb46, State: Running, Role: LEADER
I20260806 06:50:43.623662 4668 sys_catalog.cc:564] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:43.623759 4671 consensus_queue.cc:237] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } }
I20260806 06:50:43.624153 4672 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e60521a5b6fc49e5bc419126acdbdb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } } }
I20260806 06:50:43.624260 4672 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:43.624153 4673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e60521a5b6fc49e5bc419126acdbdb46. Latest consensus state: current_term: 1 leader_uuid: "e60521a5b6fc49e5bc419126acdbdb46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e60521a5b6fc49e5bc419126acdbdb46" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39795 } } }
I20260806 06:50:43.624380 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4600
I20260806 06:50:43.624403 4673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:43.624468 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:43.624641 4062 external_mini_cluster.cc:1428] Setting key cdd80e323f9c2b9a3429cde5b152b5d8
I20260806 06:50:43.624781 4675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:43.625373 4675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:43.627185 4675 catalog_manager.cc:1347] Generated new cluster ID: d15fbb13e99944a0be8e96c047ace7b4
I20260806 06:50:43.627244 4675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:43.635999 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:43.625330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.3.247.190:39795)
0806 06:50:43.626333 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:43.626343 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:43.626377 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:50:43.626502 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:43.626509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:43.626571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:43.626722 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:43.627414 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.628333 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.629147 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.629623 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.629774 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:43.632316 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:43.632321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:43.632322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:43.632337 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:43.634323 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.634705 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.634709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.634712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.634828 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.635049 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.635050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.635051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.635331 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:43.635515 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:43.635616 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:50:43.635772 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":87,"threads_started":1}
I20260806 06:50:43.665076 4675 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:43.665863 4675 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:43.672570 4675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e60521a5b6fc49e5bc419126acdbdb46: Generated new TSK 0
I20260806 06:50:43.672770 4675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:43.706938 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:43.790247 4694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:43.790481 4694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:43.790509 4694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:43.791882 4694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:43.791935 4694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:43.791967 4694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:43.793706 4694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:43.794094 4694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:43.794382 4694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:43.799885 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.799985 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.800117 4702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:43.799994 4694 server_base.cc:1034] running on GCE node
I20260806 06:50:43.800403 4694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:43.800702 4694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:43.801868 4694 hybrid_clock.cc:648] HybridClock initialized: now 1785999043801851 us; error 51 us; skew 500 ppm
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:43.804514 4694 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:43.805069 4694 webserver.cc:460] Webserver started at http://127.3.247.129:36123/ using document root <none> and password file <none>
I20260806 06:50:43.805328 4694 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:43.805399 4694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:43.805554 4694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:43.806726 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "f5972bf145f44c2180c48302ad46c422"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "4083e4eb6f4ec32f64909e74fe41a3ae"
server_key_iv: "a87bf699bc2a3c458f102ed2872574e7"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.807123 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "f5972bf145f44c2180c48302ad46c422"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "4083e4eb6f4ec32f64909e74fe41a3ae"
server_key_iv: "a87bf699bc2a3c458f102ed2872574e7"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.808559 4694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:43.809365 4709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.809542 4694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:43.809652 4694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "f5972bf145f44c2180c48302ad46c422"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "4083e4eb6f4ec32f64909e74fe41a3ae"
server_key_iv: "a87bf699bc2a3c458f102ed2872574e7"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.809736 4694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:43.831555 4694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:43.833503 4694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:43.833673 4694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:43.833974 4694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:43.834300 4694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:43.834355 4694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.834396 4694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:43.834426 4694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.839975 4694 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:42235
I20260806 06:50:43.840018 4822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:42235 every 8 connection(s)
I20260806 06:50:43.840325 4694 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:43.841686 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4694
I20260806 06:50:43.841802 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:43.842005 4062 external_mini_cluster.cc:1428] Setting key 6aa9cec14564e9054ebab45ed46b8984
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:50:43.850838 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:43.841057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:37869 (local address 127.3.247.190:39795)
0806 06:50:43.841112 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:43.841114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:43.841326 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:50:43.842666 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:43.842667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:43.842683 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:43.842753 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:43.843614 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.844154 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.845191 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.845435 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.845512 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:43.847958 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:43.847959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:43.847959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:43.847966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:43.849380 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.849828 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.849829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.849829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.849892 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.850178 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.850179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.850180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.850296 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:43.850371 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:43.850652 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 06:50:43.850747 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:50:43.851018 4825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:43.841026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39795 (local address 127.3.247.129:37869)
0806 06:50:43.841198 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:43.841240 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:43.842527 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:43.842835 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:43.842840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:43.843233 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:43.843521 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:43.843527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.844251 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.844252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:43.845092 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:43.845094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.845532 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.845533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:43.845658 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:43.846258 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:43.846266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:43.847843 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:43.849491 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:43.849498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:43.849500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:43.849724 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:43.849967 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:43.849968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:43.849968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:43.850118 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:43.850371 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:43.850378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:43.850516 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 06:50:43.850614 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:50:43.850772 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260806 06:50:43.851568 4823 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39795
I20260806 06:50:43.851651 4823 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:43.851882 4823 heartbeater.cc:511] Master 127.3.247.190:39795 requested a full tablet report, sending...
I20260806 06:50:43.852474 4632 ts_manager.cc:194] Registered new tserver with Master: f5972bf145f44c2180c48302ad46c422 (127.3.247.129:42235)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
I20260806 06:50:43.853127 4632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:37869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:43.869972 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:43.953697 4828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:43.954087 4828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:43.954114 4828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:43.955317 4828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:43.955358 4828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:43.955386 4828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:43.956848 4828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:43.957049 4828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:43.957224 4828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:43.962365 4834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.962473 4836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:43.962567 4833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:43.962551 4828 server_base.cc:1034] running on GCE node
I20260806 06:50:43.962931 4828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:43.963179 4828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:43.964344 4828 hybrid_clock.cc:648] HybridClock initialized: now 1785999043964336 us; error 35 us; skew 500 ppm
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:43.966926 4828 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:43.967432 4828 webserver.cc:460] Webserver started at http://127.3.247.130:42821/ using document root <none> and password file <none>
I20260806 06:50:43.967662 4828 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:43.967707 4828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:43.967865 4828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:43.968930 4828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "d8682537fe54419c9062f3f904f2ca9a"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "022abb56c2a09b082ae5bd16d5f2a3ff"
server_key_iv: "17e8d4cbfed93baa85b2bfb9b4a17d7c"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.969282 4828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "d8682537fe54419c9062f3f904f2ca9a"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "022abb56c2a09b082ae5bd16d5f2a3ff"
server_key_iv: "17e8d4cbfed93baa85b2bfb9b4a17d7c"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.970696 4828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:43.971483 4843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.971664 4828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:43.971758 4828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "d8682537fe54419c9062f3f904f2ca9a"
format_stamp: "Formatted at 2026-08-06 06:50:43 on dist-test-slave-0kls"
server_key: "022abb56c2a09b082ae5bd16d5f2a3ff"
server_key_iv: "17e8d4cbfed93baa85b2bfb9b4a17d7c"
server_key_version: "encryptionkey@0"
I20260806 06:50:43.971830 4828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:43.980706 4828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:43.983014 4828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:43.983175 4828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:43.983409 4828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:43.983716 4828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:43.983754 4828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.983776 4828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:43.983789 4828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:43.989573 4828 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:38471
I20260806 06:50:43.989612 4956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:38471 every 8 connection(s)
I20260806 06:50:43.989982 4828 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:43.995010 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4828
I20260806 06:50:43.995103 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:43.995292 4062 external_mini_cluster.cc:1428] Setting key 2800917ce88ab12200cf973cffd889d5
Aug 06 06:50:43 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:43.999843 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:43.990665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:42009 (local address 127.3.247.190:39795)
0806 06:50:43.990712 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:43.990714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:43.990891 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:50:43.991803 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:43.991804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:43.991814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:43.991867 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:43.992440 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.993053 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.994155 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.994390 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.994453 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:43.996841 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:43.996843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:43.996843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:43.996851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:43.998475 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.998872 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.998873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.998873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.998940 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:43.999163 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:43.999164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:43.999165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:43.999313 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:43.999381 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:43.999615 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 06:50:43.999708 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:50:43.999897 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:43.990613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39795 (local address 127.3.247.130:42009)
0806 06:50:43.990769 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:43.990817 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:43.991713 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:43.991928 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:43.991931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:43.992176 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:43.992370 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:43.992374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.993139 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.993140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:43.994074 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:43.994077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:43.994469 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:43.994470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:43.994565 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:43.995157 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:43.995166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:43.996740 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:43.998552 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:43.998558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:43.998560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:43.998773 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:43.999008 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:43.999009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:43.999009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:43.999101 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:43.999375 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:43.999379 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:43.999485 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 06:50:43.999565 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:50:43.999681 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260806 06:50:44.000680 4957 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39795
I20260806 06:50:44.000799 4957 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:44.001003 4957 heartbeater.cc:511] Master 127.3.247.190:39795 requested a full tablet report, sending...
I20260806 06:50:44.001453 4632 ts_manager.cc:194] Registered new tserver with Master: d8682537fe54419c9062f3f904f2ca9a (127.3.247.130:38471)
I20260806 06:50:44.001940 4632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:42009
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:44.025472 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:44.107815 4962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:44.108067 4962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:44.108109 4962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:44.109299 4962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:44.109377 4962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:44.109426 4962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:44.110972 4962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:44.111241 4962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:44.111480 4962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:44.116638 4967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:44.116707 4962 server_base.cc:1034] running on GCE node
W20260806 06:50:44.116636 4970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:44.116636 4968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:44.117070 4962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:44.117312 4962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:44.118448 4962 hybrid_clock.cc:648] HybridClock initialized: now 1785999044118437 us; error 27 us; skew 500 ppm
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:44.120886 4962 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:44.121349 4962 webserver.cc:460] Webserver started at http://127.3.247.131:40347/ using document root <none> and password file <none>
I20260806 06:50:44.121577 4962 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:44.121644 4962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:44.121834 4962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:44.122845 4962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "6bab99905f0d4283aaf15eacf9e4d4fd"
format_stamp: "Formatted at 2026-08-06 06:50:44 on dist-test-slave-0kls"
server_key: "233632e53132be872b1d1ff6b6aaddba"
server_key_iv: "8b7efdc6bc0a1923c691f6d628a2697b"
server_key_version: "encryptionkey@0"
I20260806 06:50:44.123209 4962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "6bab99905f0d4283aaf15eacf9e4d4fd"
format_stamp: "Formatted at 2026-08-06 06:50:44 on dist-test-slave-0kls"
server_key: "233632e53132be872b1d1ff6b6aaddba"
server_key_iv: "8b7efdc6bc0a1923c691f6d628a2697b"
server_key_version: "encryptionkey@0"
I20260806 06:50:44.124675 4962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:44.125452 4977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:44.125630 4962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:44.125730 4962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "6bab99905f0d4283aaf15eacf9e4d4fd"
format_stamp: "Formatted at 2026-08-06 06:50:44 on dist-test-slave-0kls"
server_key: "233632e53132be872b1d1ff6b6aaddba"
server_key_iv: "8b7efdc6bc0a1923c691f6d628a2697b"
server_key_version: "encryptionkey@0"
I20260806 06:50:44.125846 4962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:44.134764 4962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:44.136636 4962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:44.136785 4962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:44.137010 4962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:44.137310 4962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:44.137344 4962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:44.137367 4962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:44.137419 4962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:44.143328 4962 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:39265
I20260806 06:50:44.143373 5090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:39265 every 8 connection(s)
I20260806 06:50:44.143672 4962 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:44.150334 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 4962
I20260806 06:50:44.150440 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:44.150601 4062 external_mini_cluster.cc:1428] Setting key 091c18cf1b1894ad013735dc9c80f790
I20260806 06:50:44.152832 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.144363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:60943 (local address 127.3.247.190:39795)
0806 06:50:44.144416 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.144419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.144623 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:50:44.145531 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.145532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.145545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.145609 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:44.146284 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.146790 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.147705 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.147879 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.147930 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.150029 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:44.150030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:44.150030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:44.150036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:44.151721 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.152062 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.152062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.152063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.152115 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.152313 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.152313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.152313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.152412 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:44.152471 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.152698 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.152769 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:44.152990 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.144351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39795 (local address 127.3.247.131:60943)
0806 06:50:44.144508 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.144541 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.145410 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.145678 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.145681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.146018 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:44.146201 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.146205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.146868 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.146869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.147634 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.147636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.147947 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.147948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.148042 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.148563 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:44.148571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:44.149951 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:44.151782 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:44.151787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:44.151788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:44.151998 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:44.152170 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:44.152170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:44.152170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:44.152260 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:44.152474 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:44.152478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:44.152583 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.152662 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.152773 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260806 06:50:44.153455 5091 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39795
I20260806 06:50:44.153528 5091 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:44.153715 5091 heartbeater.cc:511] Master 127.3.247.190:39795 requested a full tablet report, sending...
I20260806 06:50:44.154125 4632 ts_manager.cc:194] Registered new tserver with Master: 6bab99905f0d4283aaf15eacf9e4d4fd (127.3.247.131:39265)
I20260806 06:50:44.154496 4632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:60943
I20260806 06:50:44.162289 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:50:44.169682 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.163780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39186 (local address 127.3.247.190:39795)
0806 06:50:44.163846 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.163849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.163933 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:50:44.164020 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.164021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.164031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.164104 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:44.164679 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.165159 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.165941 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.166127 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.166210 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.167322 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:44.167323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:44.167324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:44.167331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:44.168647 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.168954 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.168954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.168955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.169004 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.169168 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.169168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.169169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.169305 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:44.169376 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.169486 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.169544 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:50:44.171861 4632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:44.173709 4632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:44.180524 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.3.247.131:39265)
0806 06:50:44.176669 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.176674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.176710 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:50:44.176780 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.176782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.176796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.176874 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:44.177102 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178246 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.179574 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180081 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180174 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180238 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.180400 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.180443 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260806 06:50:44.180519 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39265 (local address 127.0.0.1:56842)
0806 06:50:44.176662 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.176671 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.176717 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.176970 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.176971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.176973 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:44.177022 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.177024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178365 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.178365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.179483 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.179485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180173 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.180260 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180273 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.180382 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.180439 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":42,"thread_start_us":71,"threads_started":1}
I20260806 06:50:44.180701 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:38471 (local address 127.0.0.1:53028)
0806 06:50:44.176473 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.176519 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.176592 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.176905 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.176908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.176910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:44.176986 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.176988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178374 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.178375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.179347 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.179349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180322 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.180392 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180395 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.180564 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.180613 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":58,"threads_started":1}
I20260806 06:50:44.180835 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42235 (local address 127.0.0.1:40624)
0806 06:50:44.176502 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.176533 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.176586 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.177111 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.177113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.177115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:44.177154 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.177155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178102 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.178103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.179425 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.179427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180586 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.180657 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180661 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.180716 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.180760 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260806 06:50:44.180945 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.3.247.130:38471)
0806 06:50:44.176728 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.176731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.176739 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:44.176777 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.176779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.176797 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.176876 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:44.177136 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178269 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.179435 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180152 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180227 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180283 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.180812 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.180865 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":82,"threads_started":1}
I20260806 06:50:44.180949 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.176260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.3.247.129:42235)
0806 06:50:44.176876 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.176881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.176891 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:50:44.176925 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.176927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.176942 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.177020 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:44.177232 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.178002 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.179502 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.180331 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.180392 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.180543 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.180751 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.180870 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":90,"threads_started":1}
I20260806 06:50:44.181707 4891 tablet_service.cc:1467] Processing CreateTablet for tablet b04f959c3b714433a3fa6da3ce20c4c9 (DEFAULT_TABLE table=test-table [id=a33bd98be3be49b5a7a75a30179a5df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:44.182165 4891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04f959c3b714433a3fa6da3ce20c4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:44.182112 5025 tablet_service.cc:1467] Processing CreateTablet for tablet b04f959c3b714433a3fa6da3ce20c4c9 (DEFAULT_TABLE table=test-table [id=a33bd98be3be49b5a7a75a30179a5df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:44.182222 4757 tablet_service.cc:1467] Processing CreateTablet for tablet b04f959c3b714433a3fa6da3ce20c4c9 (DEFAULT_TABLE table=test-table [id=a33bd98be3be49b5a7a75a30179a5df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:44.182443 5025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04f959c3b714433a3fa6da3ce20c4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:44.182539 4757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04f959c3b714433a3fa6da3ce20c4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:44.185089 5108 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Bootstrap starting.
I20260806 06:50:44.185580 5109 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Bootstrap starting.
I20260806 06:50:44.185865 5108 tablet_bootstrap.cc:654] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:44.185959 5110 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Bootstrap starting.
I20260806 06:50:44.186307 5109 tablet_bootstrap.cc:654] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:44.186478 5108 log.cc:824] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:44.186540 5110 tablet_bootstrap.cc:654] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:44.186868 5109 log.cc:824] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:44.187117 5110 log.cc:824] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:44.187336 5108 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: No bootstrap required, opened a new log
I20260806 06:50:44.187410 5108 ts_tablet_manager.cc:1397] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:50:44.187671 5109 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: No bootstrap required, opened a new log
I20260806 06:50:44.187740 5109 ts_tablet_manager.cc:1397] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:44.188155 5110 tablet_bootstrap.cc:492] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: No bootstrap required, opened a new log
I20260806 06:50:44.188222 5110 ts_tablet_manager.cc:1397] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:44.189191 5109 raft_consensus.cc:348] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.189371 5109 raft_consensus.cc:374] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:44.189417 5109 raft_consensus.cc:729] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8682537fe54419c9062f3f904f2ca9a, State: Initialized, Role: FOLLOWER
I20260806 06:50:44.189529 5109 consensus_queue.cc:260] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.189458 5108 raft_consensus.cc:348] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.189625 5108 raft_consensus.cc:374] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:44.189664 5108 raft_consensus.cc:729] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bab99905f0d4283aaf15eacf9e4d4fd, State: Initialized, Role: FOLLOWER
I20260806 06:50:44.189831 5109 ts_tablet_manager.cc:1428] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:44.189841 5108 consensus_queue.cc:260] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.189953 4957 heartbeater.cc:503] Master 127.3.247.190:39795 was elected leader, sending a full tablet report...
I20260806 06:50:44.190125 5091 heartbeater.cc:503] Master 127.3.247.190:39795 was elected leader, sending a full tablet report...
I20260806 06:50:44.190126 5108 ts_tablet_manager.cc:1428] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:50:44.190222 5110 raft_consensus.cc:348] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.190404 5110 raft_consensus.cc:374] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:44.190451 5110 raft_consensus.cc:729] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5972bf145f44c2180c48302ad46c422, State: Initialized, Role: FOLLOWER
I20260806 06:50:44.190553 5110 consensus_queue.cc:260] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.190843 5110 ts_tablet_manager.cc:1428] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:50:44.190928 4823 heartbeater.cc:503] Master 127.3.247.190:39795 was elected leader, sending a full tablet report...
W20260806 06:50:44.240577 4958 tablet.cc:2369] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:50:44.341009 4824 tablet.cc:2369] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:50:44.394199 5092 tablet.cc:2369] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:44.457862 5115 raft_consensus.cc:482] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:44.457991 5115 raft_consensus.cc:504] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.458473 5115 leader_election.cc:288] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8682537fe54419c9062f3f904f2ca9a (127.3.247.130:38471), f5972bf145f44c2180c48302ad46c422 (127.3.247.129:42235)
I20260806 06:50:44.461361 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.458856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:52785 (local address 127.3.247.130:38471)
0806 06:50:44.458913 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.458915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.458924 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:44.458952 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.458953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.458966 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.459021 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:44.459192 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.459687 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.460628 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.461003 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.461094 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.461103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.461254 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.461290 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:50:44.461361 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.458662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:38471 (local address 127.3.247.131:52785)
0806 06:50:44.458716 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.458728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.458797 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.459025 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.459026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.459027 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:44.459104 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.459106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.459780 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.459781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.460527 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.460529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.461089 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.461090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.461172 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.461175 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.461246 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.461289 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:50:44.461524 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.458826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:51395 (local address 127.3.247.129:42235)
0806 06:50:44.458879 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.458881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.459119 (+ 238us) server_negotiation.cc:407] Connection header received
0806 06:50:44.459191 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.459192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.459201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.459267 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:44.459420 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.459876 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.460829 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.461182 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.461268 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.461275 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.461423 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.461457 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:50:44.461536 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.458670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42235 (local address 127.3.247.131:51395)
0806 06:50:44.459040 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:44.459050 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:44.459129 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:44.459271 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:44.459272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:44.459273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:44.459344 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.459348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.459977 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.459978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.460734 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:44.460736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.461268 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.461269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:44.461348 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.461353 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:50:44.461418 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:50:44.461456 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":178,"thread_start_us":122,"threads_started":1}
I20260806 06:50:44.461740 4911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04f959c3b714433a3fa6da3ce20c4c9" candidate_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8682537fe54419c9062f3f904f2ca9a" is_pre_election: true
I20260806 06:50:44.461954 4911 raft_consensus.cc:2454] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bab99905f0d4283aaf15eacf9e4d4fd in term 0.
I20260806 06:50:44.461917 4777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04f959c3b714433a3fa6da3ce20c4c9" candidate_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5972bf145f44c2180c48302ad46c422" is_pre_election: true
I20260806 06:50:44.462046 4777 raft_consensus.cc:2454] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bab99905f0d4283aaf15eacf9e4d4fd in term 0.
I20260806 06:50:44.462189 4980 leader_election.cc:302] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bab99905f0d4283aaf15eacf9e4d4fd, d8682537fe54419c9062f3f904f2ca9a; no voters:
I20260806 06:50:44.462342 5115 raft_consensus.cc:2781] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:44.462390 5115 raft_consensus.cc:482] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:44.462419 5115 raft_consensus.cc:3037] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:44.463124 5115 raft_consensus.cc:504] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.463297 5115 leader_election.cc:288] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [CANDIDATE]: Term 1 election: Requested vote from peers d8682537fe54419c9062f3f904f2ca9a (127.3.247.130:38471), f5972bf145f44c2180c48302ad46c422 (127.3.247.129:42235)
I20260806 06:50:44.463466 4777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04f959c3b714433a3fa6da3ce20c4c9" candidate_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5972bf145f44c2180c48302ad46c422"
I20260806 06:50:44.463479 4911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04f959c3b714433a3fa6da3ce20c4c9" candidate_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8682537fe54419c9062f3f904f2ca9a"
I20260806 06:50:44.463539 4777 raft_consensus.cc:3037] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:44.463565 4911 raft_consensus.cc:3037] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:44.464258 4777 raft_consensus.cc:2454] T b04f959c3b714433a3fa6da3ce20c4c9 P f5972bf145f44c2180c48302ad46c422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bab99905f0d4283aaf15eacf9e4d4fd in term 1.
I20260806 06:50:44.464385 4911 raft_consensus.cc:2454] T b04f959c3b714433a3fa6da3ce20c4c9 P d8682537fe54419c9062f3f904f2ca9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bab99905f0d4283aaf15eacf9e4d4fd in term 1.
I20260806 06:50:44.464491 4979 leader_election.cc:302] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bab99905f0d4283aaf15eacf9e4d4fd, f5972bf145f44c2180c48302ad46c422; no voters:
I20260806 06:50:44.464617 5115 raft_consensus.cc:2781] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:44.464774 5115 raft_consensus.cc:686] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [term 1 LEADER]: Becoming Leader. State: Replica: 6bab99905f0d4283aaf15eacf9e4d4fd, State: Running, Role: LEADER
I20260806 06:50:44.464872 5115 consensus_queue.cc:237] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } }
I20260806 06:50:44.465670 4632 catalog_manager.cc:5495] T b04f959c3b714433a3fa6da3ce20c4c9 P 6bab99905f0d4283aaf15eacf9e4d4fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bab99905f0d4283aaf15eacf9e4d4fd (127.3.247.131). New cstate: current_term: 1 leader_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8682537fe54419c9062f3f904f2ca9a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5972bf145f44c2180c48302ad46c422" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bab99905f0d4283aaf15eacf9e4d4fd" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39265 } health_report { overall_health: HEALTHY } } }
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:44.537135 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.529849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.3.247.129:42235)
0806 06:50:44.529918 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.529920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.530033 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:50:44.530114 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.530116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.530127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.530170 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:44.530738 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.531226 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.531976 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.532186 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.532264 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.534513 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:44.534526 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:44.534527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:44.534534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:44.535944 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.536306 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.536310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.536310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.536378 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.536576 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.536577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.536578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.536724 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:44.536890 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.536930 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.537013 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:44.559146 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.552375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.3.247.129:42235)
0806 06:50:44.552444 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.552446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.552515 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:50:44.552565 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.552566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.552575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.552624 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:44.553130 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.553614 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.554336 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.554533 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.554586 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.556884 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:44.556886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:44.556886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:44.556893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:44.558224 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.558534 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.558535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.558535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.558583 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.558732 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.558733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.558733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.558815 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:44.558882 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.559022 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.559081 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[4587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:44.581162 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:44.574083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.3.247.129:42235)
0806 06:50:44.574167 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:44.574169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:44.574267 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:50:44.574368 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:44.574369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:44.574381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:44.574471 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:44.575051 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.575607 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.576332 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:44.576545 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:44.576624 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:44.578912 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:44.578915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:44.578915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:44.578922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:44.580212 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.580534 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.580535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.580535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.580584 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:44.580770 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:44.580771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:44.580771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:44.580860 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:44.580933 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:44.581032 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:50:44.581094 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 06:50:44.581671 4737 tablet_service.cc:2908] Rejecting scan request for tablet b04f959c3b714433a3fa6da3ce20c4c9: Uninitialized: clean time has not yet been initialized
I20260806 06:50:44.582574 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4694
I20260806 06:50:44.587472 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4828
I20260806 06:50:44.592063 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4962
I20260806 06:50:44.596654 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 4600
2026-08-06T06:50:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2312 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[5132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[5132](info): set up 2 sockets
Aug 06 06:50:44 dist-test-slave-0kls krb5kdc[5132](info): commencing operation
krb5kdc: starting...
W20260806 06:50:45.629329 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:45 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:45.748744 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38925
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:45.829716 5145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:45.830019 5145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:45.830044 5145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:45.831182 5145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:45.831214 5145 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:50:45.831228 5145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:45.831240 5145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:45.831254 5145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:45.831265 5145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:45.832674 5145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38925
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:45.832863 5145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:45.833046 5145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:45.837913 5151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:45.837940 5150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:45.837975 5145 server_base.cc:1034] running on GCE node
W20260806 06:50:45.837940 5153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:45.838400 5145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:45.838666 5145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:45.839865 5145 hybrid_clock.cc:648] HybridClock initialized: now 1785999045839857 us; error 36 us; skew 500 ppm
Aug 06 06:50:45 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999045, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:45.848336 5145 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:45.848839 5145 webserver.cc:460] Webserver started at http://127.3.247.190:37217/ using document root <none> and password file <none>
I20260806 06:50:45.849108 5145 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:45.849182 5145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:45.849334 5145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:45.850509 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "4181e68d1ae2480fa6b7741f874057ee"
format_stamp: "Formatted at 2026-08-06 06:50:45 on dist-test-slave-0kls"
server_key: "5cefeec26172d58c5cecb243850e7b0c"
server_key_iv: "807e4fd6dc6e7eab20c8a2f700e7d3cc"
server_key_version: "encryptionkey@0"
I20260806 06:50:45.850885 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "4181e68d1ae2480fa6b7741f874057ee"
format_stamp: "Formatted at 2026-08-06 06:50:45 on dist-test-slave-0kls"
server_key: "5cefeec26172d58c5cecb243850e7b0c"
server_key_iv: "807e4fd6dc6e7eab20c8a2f700e7d3cc"
server_key_version: "encryptionkey@0"
I20260806 06:50:45.852236 5145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:45.852999 5160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:45.853173 5145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:45.853267 5145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "4181e68d1ae2480fa6b7741f874057ee"
format_stamp: "Formatted at 2026-08-06 06:50:45 on dist-test-slave-0kls"
server_key: "5cefeec26172d58c5cecb243850e7b0c"
server_key_iv: "807e4fd6dc6e7eab20c8a2f700e7d3cc"
server_key_version: "encryptionkey@0"
I20260806 06:50:45.853348 5145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:45.877211 5145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:45.879290 5145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:45.879473 5145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:45.884095 5145 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38925
I20260806 06:50:45.884146 5222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38925 every 8 connection(s)
I20260806 06:50:45.884477 5145 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:45.885094 5223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:45.887411 5223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Bootstrap starting.
I20260806 06:50:45.888006 5223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:45.888300 5223 log.cc:824] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:45.889047 5223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: No bootstrap required, opened a new log
I20260806 06:50:45.891142 5223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } }
I20260806 06:50:45.891307 5223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:45.891358 5223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4181e68d1ae2480fa6b7741f874057ee, State: Initialized, Role: FOLLOWER
I20260806 06:50:45.891541 5223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } }
I20260806 06:50:45.891625 5223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:45.891669 5223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:45.891719 5223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:45.892242 5223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } }
I20260806 06:50:45.892359 5223 leader_election.cc:302] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4181e68d1ae2480fa6b7741f874057ee; no voters:
I20260806 06:50:45.892565 5223 leader_election.cc:288] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:45.892621 5226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:45.892797 5226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [term 1 LEADER]: Becoming Leader. State: Replica: 4181e68d1ae2480fa6b7741f874057ee, State: Running, Role: LEADER
I20260806 06:50:45.892827 5223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:45.892886 5226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } }
I20260806 06:50:45.893296 5228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4181e68d1ae2480fa6b7741f874057ee. Latest consensus state: current_term: 1 leader_uuid: "4181e68d1ae2480fa6b7741f874057ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } } }
I20260806 06:50:45.893368 5228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:45.893272 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4181e68d1ae2480fa6b7741f874057ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181e68d1ae2480fa6b7741f874057ee" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38925 } } }
I20260806 06:50:45.893515 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:45.893513 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5145
I20260806 06:50:45.893597 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:45.893790 4062 external_mini_cluster.cc:1428] Setting key 76c5c4e84b58ffa676c69869af245126
I20260806 06:50:45.893889 5232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:45.894423 5232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:45.896063 5232 catalog_manager.cc:1347] Generated new cluster ID: 792631e86d6c48c5b07d423971d60968
I20260806 06:50:45.896117 5232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:45 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:45.904515 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:45.894270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.3.247.190:38925)
0806 06:50:45.895400 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:45.895410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:45.895442 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:50:45.895561 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:45.895568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:45.895624 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:45.895800 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:45.896491 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:45.897419 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:45.898263 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:45.898692 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:45.898822 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:45.901118 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:45.901121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:45.901122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:45.901135 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:45.902892 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:45.903281 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:45.903284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:45.903284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:45.903382 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:45.903599 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:45.903600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:45.903600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:45.903871 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:45.904063 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:45.904134 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:50:45.904288 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":83,"threads_started":1}
I20260806 06:50:45.912300 5232 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:45.912833 5232 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:45.927160 5232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Generated new TSK 0
I20260806 06:50:45.927372 5232 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:45.956296 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38925
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:46.041061 5249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:46.041270 5249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:46.041296 5249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:46.042593 5249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:46.042635 5249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:46.042656 5249 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:46.042711 5249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:46.044165 5249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:46.044430 5249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:46.044663 5249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:46.049682 5255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:46.049708 5249 server_base.cc:1034] running on GCE node
W20260806 06:50:46.049736 5257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:46.049963 5254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:46.050202 5249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:46.050465 5249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:46.051628 5249 hybrid_clock.cc:648] HybridClock initialized: now 1785999046051623 us; error 37 us; skew 500 ppm
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:46.054069 5249 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:46.054551 5249 webserver.cc:460] Webserver started at http://127.3.247.129:43821/ using document root <none> and password file <none>
I20260806 06:50:46.054790 5249 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:46.054858 5249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:46.055018 5249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:46.056159 5249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "049fc829e3b240e4bf101b2a22a3864a"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "c3374cea1e2e3826ea9d47870c4bcd99"
server_key_iv: "b3f51ae8399dfb93d1b09a5005e27d49"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.056538 5249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "049fc829e3b240e4bf101b2a22a3864a"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "c3374cea1e2e3826ea9d47870c4bcd99"
server_key_iv: "b3f51ae8399dfb93d1b09a5005e27d49"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.058009 5249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:46.058786 5264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.058964 5249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:46.059069 5249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "049fc829e3b240e4bf101b2a22a3864a"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "c3374cea1e2e3826ea9d47870c4bcd99"
server_key_iv: "b3f51ae8399dfb93d1b09a5005e27d49"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.059146 5249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:46.070079 5249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:46.071980 5249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:46.072149 5249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:46.072777 5249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:46.072827 5249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.072868 5249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:46.072914 5249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:46.080163 5249 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:43391
I20260806 06:50:46.080260 5379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:43391 every 8 connection(s)
I20260806 06:50:46.080621 5249 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:46.081470 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5249
I20260806 06:50:46.081545 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:46.081741 4062 external_mini_cluster.cc:1428] Setting key e91d66c03404120cc0b76dad2661e7b3
I20260806 06:50:46.082834 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.073141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49317 (local address 127.3.247.190:38925)
0806 06:50:46.073208 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.073211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.073447 (+ 236us) server_negotiation.cc:407] Connection header received
0806 06:50:46.074000 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.074001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.074019 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.074099 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.074782 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.075639 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.076629 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.077015 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.077103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.079604 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.079605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.079606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.079623 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.081288 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.081728 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.081730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.081730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.081817 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.082109 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.082109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.082110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.082228 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.082323 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.082638 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.082729 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:50:46.083004 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.073153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.129:49317)
0806 06:50:46.073312 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.073354 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.073902 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.074194 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.074199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.074510 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.074701 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.074706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.075739 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.075740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.076563 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.076565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.077128 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.077129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.077254 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.077807 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.077819 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.079501 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.081356 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.081361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.081362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.081641 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.081897 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.081899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.081899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.082043 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.082313 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.082320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.082485 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.082615 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.082772 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260806 06:50:46.084077 5380 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38925
I20260806 06:50:46.084161 5380 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:46.084362 5380 heartbeater.cc:511] Master 127.3.247.190:38925 requested a full tablet report, sending...
I20260806 06:50:46.085273 5176 ts_manager.cc:194] Registered new tserver with Master: 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391)
I20260806 06:50:46.086261 5176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:49317
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:50:46.092391 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.085096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.129:58639)
0806 06:50:46.086509 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.086521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.086591 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.086716 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.086716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.086938 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.087031 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.087034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.087668 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.087670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.088595 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.088598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.088930 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.088931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.089005 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.089421 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.089428 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.089935 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.091560 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.091561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.091562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.091739 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.091933 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.091933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.091934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.091984 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.092155 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.092156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.092188 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.092239 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.092314 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1385,"thread_start_us":63,"threads_started":1}
I20260806 06:50:46.092532 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.085141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:58639 (local address 127.3.247.190:38925)
0806 06:50:46.085177 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.085179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.086575 (+ 1396us) server_negotiation.cc:407] Connection header received
0806 06:50:46.086639 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.086640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.086652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.086698 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.087107 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.087566 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.088677 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.088849 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.088910 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.090018 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.090019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.090019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.090025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.091460 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.091806 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.091806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.091807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.091856 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.092023 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.092023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.092023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.092108 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.092151 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.092253 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.092370 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:46.108314 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38925
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:46.188727 5389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:46.188943 5389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:46.188967 5389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:46.190176 5389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:46.190219 5389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:46.190240 5389 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:46.190294 5389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:46.191705 5389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:46.191948 5389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:46.192229 5389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:46.197497 5389 server_base.cc:1034] running on GCE node
W20260806 06:50:46.197479 5395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:46.197531 5397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:46.197639 5394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:46.197942 5389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:46.198148 5389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:46.199301 5389 hybrid_clock.cc:648] HybridClock initialized: now 1785999046199287 us; error 29 us; skew 500 ppm
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:46.202525 5389 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:46.202996 5389 webserver.cc:460] Webserver started at http://127.3.247.130:33825/ using document root <none> and password file <none>
I20260806 06:50:46.203238 5389 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:46.203282 5389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:46.203431 5389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:46.204483 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "083c80c6b85c4ae087cbaf1613fc3e87"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "8b17522defd87c984c11bfcadabac055"
server_key_iv: "64fc07b5a1f19845fb7de63896fa0106"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.204830 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "083c80c6b85c4ae087cbaf1613fc3e87"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "8b17522defd87c984c11bfcadabac055"
server_key_iv: "64fc07b5a1f19845fb7de63896fa0106"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.206271 5389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:46.206955 5404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.207109 5389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:46.207202 5389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "083c80c6b85c4ae087cbaf1613fc3e87"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "8b17522defd87c984c11bfcadabac055"
server_key_iv: "64fc07b5a1f19845fb7de63896fa0106"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.207278 5389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:46.219965 5389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:46.221885 5389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:46.222043 5389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:46.222599 5389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:46.222664 5389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.222694 5389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:46.222712 5389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:46.229970 5389 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:38769
I20260806 06:50:46.230098 5519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:38769 every 8 connection(s)
I20260806 06:50:46.230377 5389 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:46.231508 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.222917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:60539 (local address 127.3.247.190:38925)
0806 06:50:46.222971 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.222979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.223203 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:50:46.223713 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.223715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.223728 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.223780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.224457 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.224962 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.226041 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.226235 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.226319 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.228652 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.228654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.228655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.228663 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.230250 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.230689 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.230691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.230691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.230752 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.231028 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.231029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.231029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.231136 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.231187 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.231374 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.231444 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:50:46.231694 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.222906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.130:60539)
0806 06:50:46.223088 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.223122 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.223608 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.223848 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.223853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.224174 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.224362 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.224367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.225055 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.225056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.225944 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.225947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.226327 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.226328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.226460 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.226964 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.226975 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.228563 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.230325 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.230330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.230332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.230602 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.230814 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.230815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.230816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.230956 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.231177 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.231179 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.231286 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.231363 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.231486 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260806 06:50:46.232146 5520 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38925
I20260806 06:50:46.232228 5520 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:46.232452 5520 heartbeater.cc:511] Master 127.3.247.190:38925 requested a full tablet report, sending...
I20260806 06:50:46.232824 5176 ts_manager.cc:194] Registered new tserver with Master: 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769)
I20260806 06:50:46.232908 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5389
I20260806 06:50:46.232968 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:46.233147 4062 external_mini_cluster.cc:1428] Setting key a13d7807c5f256b2663b95e0f090ea7f
I20260806 06:50:46.233248 5176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:60539
I20260806 06:50:46.239475 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.233353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:46977 (local address 127.3.247.190:38925)
0806 06:50:46.233403 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.233405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.233413 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:46.233461 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.233462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.233471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.233526 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.233920 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.234547 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.235576 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.235886 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.235959 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.237001 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.237002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.237003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.237008 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.238344 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.238741 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.238742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.238742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.238788 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.238998 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.238998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.238998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.239121 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.239170 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.239359 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.239414 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:46.239601 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.233192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.130:46977)
0806 06:50:46.233342 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.233354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.233410 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.233531 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.233531 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.233731 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.233820 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.233822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.234649 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.234650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.235497 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.235499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.235975 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.235976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.236029 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.236478 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.236485 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.236908 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.238426 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.238427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.238428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.238665 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.238854 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.238855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.238856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.238935 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.239186 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.239187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.239240 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.239337 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.239451 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:46.258492 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38925
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:46.339185 5529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:46.339434 5529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:46.339480 5529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:46.340693 5529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:46.340759 5529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:46.340798 5529 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:46.340838 5529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:46.342305 5529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:46.342628 5529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:46.342864 5529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:46.347786 5537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:46.347815 5529 server_base.cc:1034] running on GCE node
W20260806 06:50:46.347973 5535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:46.348150 5534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:46.348408 5529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:46.348629 5529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:46.349774 5529 hybrid_clock.cc:648] HybridClock initialized: now 1785999046349764 us; error 27 us; skew 500 ppm
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:46.352300 5529 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:46.352821 5529 webserver.cc:460] Webserver started at http://127.3.247.131:44423/ using document root <none> and password file <none>
I20260806 06:50:46.353080 5529 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:46.353150 5529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:46.353302 5529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:46.354435 5529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "34dd449d08764be182ddfe1d205cc9b5"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "bf56c8e8bb34fe96c854093c9a0a8771"
server_key_iv: "1f05dc21f5f4fe41a03992ccd243dd4d"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.354792 5529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "34dd449d08764be182ddfe1d205cc9b5"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "bf56c8e8bb34fe96c854093c9a0a8771"
server_key_iv: "1f05dc21f5f4fe41a03992ccd243dd4d"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.356114 5529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:46.356933 5544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.357120 5529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:46.357210 5529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "34dd449d08764be182ddfe1d205cc9b5"
format_stamp: "Formatted at 2026-08-06 06:50:46 on dist-test-slave-0kls"
server_key: "bf56c8e8bb34fe96c854093c9a0a8771"
server_key_iv: "1f05dc21f5f4fe41a03992ccd243dd4d"
server_key_version: "encryptionkey@0"
I20260806 06:50:46.357277 5529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:46.368193 5529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:46.370775 5529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:46.370910 5529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:46.371383 5529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:46.371428 5529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.371451 5529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:46.371500 5529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:46.378857 5529 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:37987
I20260806 06:50:46.379014 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:37987 every 8 connection(s)
I20260806 06:50:46.379293 5529 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:50:46.381839 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.371896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:60475 (local address 127.3.247.190:38925)
0806 06:50:46.371975 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.371978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.372144 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:50:46.372653 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.372655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.372670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.372757 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.373585 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.374471 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.375591 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.375935 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.376039 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.378672 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.378674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.378674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.378683 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.380348 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.380788 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.380790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.380790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.380863 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.381129 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.381130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.381130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.381257 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.381333 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.381619 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.381715 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:50:46.382053 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.371854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.131:60475)
0806 06:50:46.372010 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.372051 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.372554 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.372847 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.372851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.373178 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.373475 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.373483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.374573 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.374574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.375517 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.375520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.376058 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.376059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.376231 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.376841 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.376856 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.378556 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.380453 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.380459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.380461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.380692 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.380935 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.380936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.380936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.381054 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.381324 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.381330 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.381480 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.381596 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.381766 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260806 06:50:46.382581 5660 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38925
I20260806 06:50:46.382666 5660 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:46.382866 5660 heartbeater.cc:511] Master 127.3.247.190:38925 requested a full tablet report, sending...
I20260806 06:50:46.383209 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5529
I20260806 06:50:46.383247 5176 ts_manager.cc:194] Registered new tserver with Master: 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131:37987)
I20260806 06:50:46.383322 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:46.383533 4062 external_mini_cluster.cc:1428] Setting key 957ce2c2911ed4bce27e2316b020ad5b
I20260806 06:50:46.383720 5176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:60475
I20260806 06:50:46.384518 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:50:46.389987 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.383846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:33045 (local address 127.3.247.190:38925)
0806 06:50:46.383881 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.383883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.383891 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:46.384020 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.384022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.384031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.384100 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.384558 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.385092 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.386272 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.386462 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.386543 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.387500 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.387501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.387502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.387508 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.388815 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.389163 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.389164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.389165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.389218 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.389449 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.389450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.389451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.389561 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.389633 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.389815 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.389887 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:50:46.389995 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.383631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.3.247.131:33045)
0806 06:50:46.383758 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.383769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.383932 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.384105 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.384106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.384350 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.384456 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.384458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.385198 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.385200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.386197 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.386199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.386542 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.386544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.386624 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.387095 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.387100 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.387396 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.388927 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.388928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.388928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.389094 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.389290 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.389291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.389292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.389371 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.389638 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.389639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.389683 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.389763 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.389890 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:46.407368 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.400820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.3.247.190:38925)
0806 06:50:46.400883 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.400885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.400894 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:46.400919 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.400920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.400935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.400984 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.401513 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.402062 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.402773 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.402946 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.402993 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.405217 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.405218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.405219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.405225 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.406441 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.406761 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.406762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.406762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.406810 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.406971 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.406971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.406972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.407067 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.407138 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.407224 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.407280 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:50:46.409188 5177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:46.410941 5177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:46.417088 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.413176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:38769 (local address 127.0.0.1:51116)
0806 06:50:46.413467 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.413479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.413524 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.413847 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.413848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.413850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.413898 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.413900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.414741 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.414743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.415992 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.415995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416785 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.416863 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.416876 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.416946 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.417003 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":33,"threads_started":1}
I20260806 06:50:46.417200 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.412967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37987 (local address 127.0.0.1:46302)
0806 06:50:46.413106 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.413153 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.413238 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.413630 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.413633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.413635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.413703 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.413705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.415073 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.415074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.415988 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.415990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416916 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.416994 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.416998 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.417030 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.417074 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260806 06:50:46.417277 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.412988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43391 (local address 127.0.0.1:58660)
0806 06:50:46.413251 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.413262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.413337 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.413696 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.413697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.413698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.413747 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.413748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.415049 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.415050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.416102 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.416104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416923 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.416983 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.416986 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.417036 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.417070 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260806 06:50:46.417359 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.413300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.3.247.130:38769)
0806 06:50:46.413627 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.413631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.413639 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:46.413675 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.413677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.413692 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.413759 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.413976 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.414653 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416088 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416687 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416782 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.416844 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.417208 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.417253 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":80,"threads_started":1}
I20260806 06:50:46.417411 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.413278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.3.247.131:37987)
0806 06:50:46.413448 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.413453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.413464 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:46.413506 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.413509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.413526 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.413610 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.413840 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.414962 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416071 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416807 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.417147 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.417215 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.417243 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.417307 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260806 06:50:46.417897 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.413082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58660 (local address 127.3.247.129:43391)
0806 06:50:46.413485 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.413489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.413497 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:46.413528 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.413530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.413541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.413632 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.413897 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.414961 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416176 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.416850 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.416918 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.416974 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.417047 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.417082 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":82,"threads_started":1}
I20260806 06:50:46.418107 5594 tablet_service.cc:1467] Processing CreateTablet for tablet 4e906152fa484ff787655f3344786d68 (DEFAULT_TABLE table=test-table [id=e7a19668f067438482d5d234ac6ab02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.418107 5454 tablet_service.cc:1467] Processing CreateTablet for tablet 4e906152fa484ff787655f3344786d68 (DEFAULT_TABLE table=test-table [id=e7a19668f067438482d5d234ac6ab02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.418452 5594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e906152fa484ff787655f3344786d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.418457 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e906152fa484ff787655f3344786d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.420881 5680 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Bootstrap starting.
I20260806 06:50:46.421490 5314 tablet_service.cc:1467] Processing CreateTablet for tablet 4e906152fa484ff787655f3344786d68 (DEFAULT_TABLE table=test-table [id=e7a19668f067438482d5d234ac6ab02f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.421833 5314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e906152fa484ff787655f3344786d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.421917 5680 tablet_bootstrap.cc:654] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.421936 5681 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Bootstrap starting.
I20260806 06:50:46.422243 5680 log.cc:824] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:46.422598 5681 tablet_bootstrap.cc:654] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.422923 5681 log.cc:824] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:46.422976 5680 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: No bootstrap required, opened a new log
I20260806 06:50:46.423056 5680 ts_tablet_manager.cc:1397] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:50:46.424573 5680 raft_consensus.cc:348] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.424726 5680 raft_consensus.cc:374] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.424765 5680 raft_consensus.cc:729] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dd449d08764be182ddfe1d205cc9b5, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.424903 5680 consensus_queue.cc:260] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.424997 5684 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Bootstrap starting.
I20260806 06:50:46.425160 5680 ts_tablet_manager.cc:1428] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:50:46.425289 5660 heartbeater.cc:503] Master 127.3.247.190:38925 was elected leader, sending a full tablet report...
I20260806 06:50:46.425592 5684 tablet_bootstrap.cc:654] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.425935 5684 log.cc:824] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:46.426102 5681 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: No bootstrap required, opened a new log
I20260806 06:50:46.426172 5681 ts_tablet_manager.cc:1397] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 06:50:46.426734 5684 tablet_bootstrap.cc:492] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: No bootstrap required, opened a new log
I20260806 06:50:46.426813 5684 ts_tablet_manager.cc:1397] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:46.427757 5681 raft_consensus.cc:348] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.427913 5681 raft_consensus.cc:374] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.427969 5681 raft_consensus.cc:729] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083c80c6b85c4ae087cbaf1613fc3e87, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.428112 5681 consensus_queue.cc:260] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.428369 5681 ts_tablet_manager.cc:1428] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:50:46.428413 5520 heartbeater.cc:503] Master 127.3.247.190:38925 was elected leader, sending a full tablet report...
I20260806 06:50:46.428520 5684 raft_consensus.cc:348] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.428658 5684 raft_consensus.cc:374] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.428704 5684 raft_consensus.cc:729] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049fc829e3b240e4bf101b2a22a3864a, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.428809 5684 consensus_queue.cc:260] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.429061 5684 ts_tablet_manager.cc:1428] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:50:46.429121 5380 heartbeater.cc:503] Master 127.3.247.190:38925 was elected leader, sending a full tablet report...
W20260806 06:50:46.481065 5521 tablet.cc:2369] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:46.510473 5687 raft_consensus.cc:482] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:46.510604 5687 raft_consensus.cc:504] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.510996 5687 leader_election.cc:288] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131:37987), 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391)
I20260806 06:50:46.513998 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.511297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:33525 (local address 127.3.247.129:43391)
0806 06:50:46.511340 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.511342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.511467 (+ 125us) server_negotiation.cc:407] Connection header received
0806 06:50:46.511508 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.511509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.511518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.511573 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.511784 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.512249 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513209 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.513576 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513639 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.513669 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.513876 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.513925 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:46.514006 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.511172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37987 (local address 127.3.247.130:40273)
0806 06:50:46.511232 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.511244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.511288 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.511484 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.511486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.511488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.511572 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.511574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.512310 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.512311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.513113 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.513115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.513701 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.513824 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.513828 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.513885 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.513919 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":11}
I20260806 06:50:46.514007 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.511284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:40273 (local address 127.3.247.131:37987)
0806 06:50:46.511341 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.511343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.511354 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:46.511385 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.511386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.511396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.511463 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.511659 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.512218 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513244 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.513592 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513681 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.513690 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.513904 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.513945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:50:46.514014 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.511194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43391 (local address 127.3.247.130:33525)
0806 06:50:46.511376 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.511399 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.511453 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.511601 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.511602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.511603 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.511709 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.511710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.512357 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.512358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.513110 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.513112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.513667 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.513668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.513754 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.513758 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.513863 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.513909 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260806 06:50:46.514376 5614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e906152fa484ff787655f3344786d68" candidate_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dd449d08764be182ddfe1d205cc9b5" is_pre_election: true
I20260806 06:50:46.514379 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e906152fa484ff787655f3344786d68" candidate_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a" is_pre_election: true
I20260806 06:50:46.514554 5334 raft_consensus.cc:2454] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083c80c6b85c4ae087cbaf1613fc3e87 in term 0.
I20260806 06:50:46.514545 5614 raft_consensus.cc:2454] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083c80c6b85c4ae087cbaf1613fc3e87 in term 0.
I20260806 06:50:46.514812 5406 leader_election.cc:302] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049fc829e3b240e4bf101b2a22a3864a, 083c80c6b85c4ae087cbaf1613fc3e87; no voters:
I20260806 06:50:46.514995 5687 raft_consensus.cc:2781] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:46.515041 5687 raft_consensus.cc:482] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:46.515057 5687 raft_consensus.cc:3037] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.515995 5687 raft_consensus.cc:504] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.516149 5687 leader_election.cc:288] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [CANDIDATE]: Term 1 election: Requested vote from peers 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131:37987), 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391)
I20260806 06:50:46.516305 5614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e906152fa484ff787655f3344786d68" candidate_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dd449d08764be182ddfe1d205cc9b5"
I20260806 06:50:46.516312 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e906152fa484ff787655f3344786d68" candidate_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a"
I20260806 06:50:46.516396 5614 raft_consensus.cc:3037] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.516402 5334 raft_consensus.cc:3037] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.517315 5614 raft_consensus.cc:2454] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083c80c6b85c4ae087cbaf1613fc3e87 in term 1.
I20260806 06:50:46.517315 5334 raft_consensus.cc:2454] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083c80c6b85c4ae087cbaf1613fc3e87 in term 1.
I20260806 06:50:46.517509 5406 leader_election.cc:302] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049fc829e3b240e4bf101b2a22a3864a, 083c80c6b85c4ae087cbaf1613fc3e87; no voters:
I20260806 06:50:46.517645 5687 raft_consensus.cc:2781] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:46.517868 5687 raft_consensus.cc:686] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 LEADER]: Becoming Leader. State: Replica: 083c80c6b85c4ae087cbaf1613fc3e87, State: Running, Role: LEADER
I20260806 06:50:46.517987 5687 consensus_queue.cc:237] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } }
I20260806 06:50:46.518831 5177 catalog_manager.cc:5495] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } health_report { overall_health: UNKNOWN } } }
I20260806 06:50:46.536698 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.534115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.3.247.130:38769)
0806 06:50:46.534162 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.534164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.534179 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:50:46.534308 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.534313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.534330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.534401 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:46.534580 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.535046 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.535833 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.536028 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.536101 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.536328 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.536431 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.536524 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.536569 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.536590 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:50:46.538455 5614 raft_consensus.cc:1261] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Refusing update from remote peer 083c80c6b85c4ae087cbaf1613fc3e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.538455 5334 raft_consensus.cc:1261] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Refusing update from remote peer 083c80c6b85c4ae087cbaf1613fc3e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.538861 5690 consensus_queue.cc:1035] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.539011 5687 consensus_queue.cc:1035] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.541918 5694 mvcc.cc:205] Tried to move back new op lower bound from 7315452094618955776 to 7315452094538338304. Current Snapshot: MvccSnapshot[applied={T|T < 7315452094618955776}]
I20260806 06:50:46.542050 5695 mvcc.cc:205] Tried to move back new op lower bound from 7315452094618955776 to 7315452094538338304. Current Snapshot: MvccSnapshot[applied={T|T < 7315452094618955776}]
I20260806 06:50:46.544938 5176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:46.545044 5176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:46.545773 5176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Sending DeleteTablet for 3 replicas of tablet 4e906152fa484ff787655f3344786d68
I20260806 06:50:46.546119 5594 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e906152fa484ff787655f3344786d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:46302
I20260806 06:50:46.546192 5314 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e906152fa484ff787655f3344786d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:58660
I20260806 06:50:46.546373 5176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:50:46.546458 5454 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e906152fa484ff787655f3344786d68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:51116
W20260806 06:50:46.546533 5176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:46.546638 5705 tablet_replica.cc:330] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: stopping tablet replica
I20260806 06:50:46.546725 5705 raft_consensus.cc:2229] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:46.546824 5705 raft_consensus.cc:2258] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.546820 5704 tablet_replica.cc:330] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: stopping tablet replica
I20260806 06:50:46.547055 5703 tablet_replica.cc:330] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: stopping tablet replica
I20260806 06:50:46.547156 5703 raft_consensus.cc:2229] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:46.547500 5703 raft_consensus.cc:2258] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.547991 5454 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c5552e27541a283791f0bc550fc7c (DEFAULT_TABLE table=test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.548108 5704 raft_consensus.cc:2229] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:46.548162 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c5552e27541a283791f0bc550fc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.548249 5704 raft_consensus.cc:2258] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.548216 5314 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c5552e27541a283791f0bc550fc7c (DEFAULT_TABLE table=test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.548365 5314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c5552e27541a283791f0bc550fc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.548686 5704 ts_tablet_manager.cc:1905] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.549603 5594 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c5552e27541a283791f0bc550fc7c (DEFAULT_TABLE table=test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:46.549702 5594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c5552e27541a283791f0bc550fc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.549749 5684 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Bootstrap starting.
I20260806 06:50:46.549805 5681 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Bootstrap starting.
I20260806 06:50:46.550154 5684 tablet_bootstrap.cc:654] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.550182 5681 tablet_bootstrap.cc:654] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.550990 5704 ts_tablet_manager.cc:1918] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:46.551065 5704 log.cc:1196] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/4e906152fa484ff787655f3344786d68
I20260806 06:50:46.551110 5680 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Bootstrap starting.
I20260806 06:50:46.548010 5705 ts_tablet_manager.cc:1905] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.551319 5704 ts_tablet_manager.cc:1939] T 4e906152fa484ff787655f3344786d68 P 049fc829e3b240e4bf101b2a22a3864a: Deleting consensus metadata
I20260806 06:50:46.551540 5680 tablet_bootstrap.cc:654] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.551710 5162 catalog_manager.cc:4841] TS 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391): tablet 4e906152fa484ff787655f3344786d68 (table test-table [id=e7a19668f067438482d5d234ac6ab02f]) successfully deleted
I20260806 06:50:46.552192 5681 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: No bootstrap required, opened a new log
I20260806 06:50:46.552239 5681 ts_tablet_manager.cc:1397] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:46.552408 5684 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: No bootstrap required, opened a new log
I20260806 06:50:46.552407 5681 raft_consensus.cc:348] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.552461 5684 ts_tablet_manager.cc:1397] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:50:46.552503 5681 raft_consensus.cc:374] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.552526 5681 raft_consensus.cc:729] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083c80c6b85c4ae087cbaf1613fc3e87, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.552575 5681 consensus_queue.cc:260] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.552659 5681 ts_tablet_manager.cc:1428] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.552626 5684 raft_consensus.cc:348] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.552699 5684 raft_consensus.cc:374] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.552726 5684 raft_consensus.cc:729] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049fc829e3b240e4bf101b2a22a3864a, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.552775 5705 ts_tablet_manager.cc:1918] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:46.552771 5684 consensus_queue.cc:260] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.552835 5705 log.cc:1196] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/4e906152fa484ff787655f3344786d68
I20260806 06:50:46.552866 5684 ts_tablet_manager.cc:1428] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:46.553143 5705 ts_tablet_manager.cc:1939] T 4e906152fa484ff787655f3344786d68 P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting consensus metadata
I20260806 06:50:46.553531 5163 catalog_manager.cc:4841] TS 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769): tablet 4e906152fa484ff787655f3344786d68 (table test-table [id=e7a19668f067438482d5d234ac6ab02f]) successfully deleted
I20260806 06:50:46.553853 5680 tablet_bootstrap.cc:492] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: No bootstrap required, opened a new log
I20260806 06:50:46.553857 5703 ts_tablet_manager.cc:1905] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.553915 5680 ts_tablet_manager.cc:1397] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 06:50:46.554200 5680 raft_consensus.cc:348] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.554296 5680 raft_consensus.cc:374] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.554318 5680 raft_consensus.cc:729] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dd449d08764be182ddfe1d205cc9b5, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.554358 5680 consensus_queue.cc:260] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.554437 5680 ts_tablet_manager.cc:1428] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.555359 5703 ts_tablet_manager.cc:1918] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:46.555454 5703 log.cc:1196] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/4e906152fa484ff787655f3344786d68
I20260806 06:50:46.555822 5703 ts_tablet_manager.cc:1939] T 4e906152fa484ff787655f3344786d68 P 34dd449d08764be182ddfe1d205cc9b5: Deleting consensus metadata
I20260806 06:50:46.556252 5163 catalog_manager.cc:4841] TS 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131:37987): tablet 4e906152fa484ff787655f3344786d68 (table test-table [id=e7a19668f067438482d5d234ac6ab02f]) successfully deleted
W20260806 06:50:46.581336 5381 tablet.cc:2369] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:50:46.630268 5661 tablet.cc:2369] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:46.663933 5685 raft_consensus.cc:482] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:46.664034 5685 raft_consensus.cc:504] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.664312 5685 leader_election.cc:288] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391), 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769)
I20260806 06:50:46.667411 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.664613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:58675 (local address 127.3.247.130:38769)
0806 06:50:46.664681 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.664683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.664694 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:46.664725 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.664726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.664736 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.664836 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.665009 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.665485 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.666675 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.667043 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.667130 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.667139 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.667301 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.667338 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:50:46.667411 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.664522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:38769 (local address 127.3.247.131:58675)
0806 06:50:46.664561 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.664573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.664618 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.664857 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.664859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.664862 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.664933 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.664935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.665577 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.665578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.666587 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.666589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.667137 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.667138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.667218 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.667229 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.667294 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.667336 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:50:46.667587 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.664688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:40973 (local address 127.3.247.129:43391)
0806 06:50:46.664771 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.664774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.664783 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:46.664811 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.664812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.664830 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.664896 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:46.665067 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.665527 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.666824 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.667181 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.667275 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.667284 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.667458 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.667496 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:50:46.667682 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.664524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43391 (local address 127.3.247.131:40973)
0806 06:50:46.664691 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.664699 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.664744 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.664900 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.664901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.664903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:46.664992 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.664994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.665613 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.665614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.666737 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.666740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.667294 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.667296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.667363 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.667373 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.667441 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.667588 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":47,"thread_start_us":62,"threads_started":1}
I20260806 06:50:46.667719 5474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c5552e27541a283791f0bc550fc7c" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" is_pre_election: true
I20260806 06:50:46.667872 5474 raft_consensus.cc:2454] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 0.
I20260806 06:50:46.667873 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c5552e27541a283791f0bc550fc7c" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a" is_pre_election: true
I20260806 06:50:46.667956 5334 raft_consensus.cc:2454] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 0.
I20260806 06:50:46.668081 5547 leader_election.cc:302] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083c80c6b85c4ae087cbaf1613fc3e87, 34dd449d08764be182ddfe1d205cc9b5; no voters:
I20260806 06:50:46.668275 5685 raft_consensus.cc:2781] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:46.668327 5685 raft_consensus.cc:482] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:46.668342 5685 raft_consensus.cc:3037] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.668984 5685 raft_consensus.cc:504] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.669128 5685 leader_election.cc:288] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 election: Requested vote from peers 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391), 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769)
I20260806 06:50:46.669272 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c5552e27541a283791f0bc550fc7c" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a"
I20260806 06:50:46.669353 5334 raft_consensus.cc:3037] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.669337 5474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c5552e27541a283791f0bc550fc7c" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083c80c6b85c4ae087cbaf1613fc3e87"
I20260806 06:50:46.669413 5474 raft_consensus.cc:3037] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.670028 5334 raft_consensus.cc:2454] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 1.
I20260806 06:50:46.670192 5474 raft_consensus.cc:2454] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 1.
I20260806 06:50:46.670220 5546 leader_election.cc:302] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049fc829e3b240e4bf101b2a22a3864a, 34dd449d08764be182ddfe1d205cc9b5; no voters:
I20260806 06:50:46.670324 5685 raft_consensus.cc:2781] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:46.670483 5685 raft_consensus.cc:686] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 1 LEADER]: Becoming Leader. State: Replica: 34dd449d08764be182ddfe1d205cc9b5, State: Running, Role: LEADER
I20260806 06:50:46.670534 5685 consensus_queue.cc:237] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.671236 5176 catalog_manager.cc:5495] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } health_report { overall_health: HEALTHY } } }
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999045, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:46.725271 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.718596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.3.247.190:38925)
0806 06:50:46.718670 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.718672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.718805 (+ 133us) server_negotiation.cc:407] Connection header received
0806 06:50:46.718865 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.718866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.718876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.718913 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.719342 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.719803 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.720429 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.720608 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.720655 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.722738 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.722739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.722740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.722745 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.724198 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.724608 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.724610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.724610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.724666 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.724857 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.724857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.724858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.724965 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.725017 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.725132 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.725203 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:50:46.725354 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.718550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38925 (local address 127.0.0.1:54246)
0806 06:50:46.718731 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.718740 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.718807 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.718926 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.718926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.719211 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:46.719271 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.719273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.719876 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.719877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.720367 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.720368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.720670 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.720671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.720711 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.721223 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:46.721228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:46.722659 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:46.724290 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.724297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.724299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.724516 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.724747 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:46.724748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:46.724748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:46.724801 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:46.725020 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:46.725025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:46.725066 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.725116 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.725251 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":46,"thread_start_us":128,"threads_started":1}
I20260806 06:50:46.727438 5176 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:54246:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:50:46.727634 5176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:46.729197 5314 tablet_service.cc:1467] Processing CreateTablet for tablet e5704e45415748d088d94177bb8e4be0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80493e4fd6b5421c92a7a7d5a80eeae9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:46.729197 5454 tablet_service.cc:1467] Processing CreateTablet for tablet e5704e45415748d088d94177bb8e4be0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80493e4fd6b5421c92a7a7d5a80eeae9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:46.729250 5594 tablet_service.cc:1467] Processing CreateTablet for tablet e5704e45415748d088d94177bb8e4be0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80493e4fd6b5421c92a7a7d5a80eeae9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:46.729372 5314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5704e45415748d088d94177bb8e4be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.729375 5594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5704e45415748d088d94177bb8e4be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.729375 5454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5704e45415748d088d94177bb8e4be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:46.730756 5684 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: Bootstrap starting.
I20260806 06:50:46.730938 5680 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: Bootstrap starting.
I20260806 06:50:46.731201 5684 tablet_bootstrap.cc:654] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.731348 5680 tablet_bootstrap.cc:654] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.731755 5681 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: Bootstrap starting.
I20260806 06:50:46.731904 5684 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: No bootstrap required, opened a new log
I20260806 06:50:46.731962 5684 ts_tablet_manager.cc:1397] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:46.732106 5681 tablet_bootstrap.cc:654] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:46.732090 5684 raft_consensus.cc:348] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.732141 5684 raft_consensus.cc:374] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.732154 5684 raft_consensus.cc:729] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049fc829e3b240e4bf101b2a22a3864a, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.732189 5684 consensus_queue.cc:260] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.732249 5684 ts_tablet_manager.cc:1428] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.732442 5688 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.732594 5688 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.732981 5681 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: No bootstrap required, opened a new log
I20260806 06:50:46.733035 5681 ts_tablet_manager.cc:1397] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:50:46.733148 5681 raft_consensus.cc:348] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.733237 5681 raft_consensus.cc:374] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.733266 5681 raft_consensus.cc:729] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083c80c6b85c4ae087cbaf1613fc3e87, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.733312 5681 consensus_queue.cc:260] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.733402 5681 ts_tablet_manager.cc:1428] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.733500 5687 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.733584 5687 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.733866 5680 tablet_bootstrap.cc:492] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: No bootstrap required, opened a new log
I20260806 06:50:46.733939 5680 ts_tablet_manager.cc:1397] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:50:46.734217 5680 raft_consensus.cc:348] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.734283 5680 raft_consensus.cc:374] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:46.734297 5680 raft_consensus.cc:729] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dd449d08764be182ddfe1d205cc9b5, State: Initialized, Role: FOLLOWER
I20260806 06:50:46.734354 5680 consensus_queue.cc:260] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.734460 5680 ts_tablet_manager.cc:1428] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:46.734542 5685 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.734645 5685 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.756589 5685 raft_consensus.cc:482] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:46.756711 5685 raft_consensus.cc:504] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.756939 5685 leader_election.cc:288] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391), 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769)
I20260806 06:50:46.757138 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5704e45415748d088d94177bb8e4be0" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a" is_pre_election: true
I20260806 06:50:46.757153 5474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5704e45415748d088d94177bb8e4be0" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" is_pre_election: true
I20260806 06:50:46.757244 5334 raft_consensus.cc:2454] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 0.
I20260806 06:50:46.757288 5474 raft_consensus.cc:2454] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 0.
I20260806 06:50:46.757413 5546 leader_election.cc:302] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049fc829e3b240e4bf101b2a22a3864a, 34dd449d08764be182ddfe1d205cc9b5; no voters:
I20260806 06:50:46.757529 5685 raft_consensus.cc:2781] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:46.757583 5685 raft_consensus.cc:482] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:46.757597 5685 raft_consensus.cc:3037] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.758297 5685 raft_consensus.cc:504] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.758443 5685 leader_election.cc:288] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 election: Requested vote from peers 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391), 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769)
I20260806 06:50:46.758589 5474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5704e45415748d088d94177bb8e4be0" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083c80c6b85c4ae087cbaf1613fc3e87"
I20260806 06:50:46.758601 5334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5704e45415748d088d94177bb8e4be0" candidate_uuid: "34dd449d08764be182ddfe1d205cc9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049fc829e3b240e4bf101b2a22a3864a"
I20260806 06:50:46.758669 5334 raft_consensus.cc:3037] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.758661 5474 raft_consensus.cc:3037] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:46.759264 5474 raft_consensus.cc:2454] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 1.
I20260806 06:50:46.759434 5334 raft_consensus.cc:2454] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34dd449d08764be182ddfe1d205cc9b5 in term 1.
I20260806 06:50:46.759446 5547 leader_election.cc:302] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083c80c6b85c4ae087cbaf1613fc3e87, 34dd449d08764be182ddfe1d205cc9b5; no voters:
I20260806 06:50:46.759542 5685 raft_consensus.cc:2781] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:46.759593 5685 raft_consensus.cc:686] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [term 1 LEADER]: Becoming Leader. State: Replica: 34dd449d08764be182ddfe1d205cc9b5, State: Running, Role: LEADER
I20260806 06:50:46.759642 5685 consensus_queue.cc:237] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } }
I20260806 06:50:46.759982 5710 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: TxnStatusTablet state changed. Reason: New leader 34dd449d08764be182ddfe1d205cc9b5. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.760066 5710 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:50:46.760255 5722 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:50:46.760267 5177 catalog_manager.cc:5495] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } health_report { overall_health: HEALTHY } } }
I20260806 06:50:46.760377 5722 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:50:46.769619 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.767215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.3.247.131:37987)
0806 06:50:46.767262 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.767264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.767272 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:46.767342 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.767342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.767361 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.767423 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:46.767553 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.768034 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.768792 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.769009 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.769085 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.769280 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.769380 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.769462 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.769518 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.769541 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:50:46.769626 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.767125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37987 (local address 127.0.0.1:46318)
0806 06:50:46.767197 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.767210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.767258 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.767439 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.767440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.767441 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:46.767491 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.767493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.768121 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.768122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.768704 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.768706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.769084 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.769085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.769152 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.769175 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.769449 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.769458 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.769513 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.769559 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:50:46.770761 5334 raft_consensus.cc:1261] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Refusing update from remote peer 34dd449d08764be182ddfe1d205cc9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.770918 5474 raft_consensus.cc:1261] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Refusing update from remote peer 34dd449d08764be182ddfe1d205cc9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.771078 5710 consensus_queue.cc:1035] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.771198 5685 consensus_queue.cc:1035] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.771740 5688 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: TxnStatusTablet state changed. Reason: New leader 34dd449d08764be182ddfe1d205cc9b5. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.771847 5688 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.771829 5687 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: TxnStatusTablet state changed. Reason: New leader 34dd449d08764be182ddfe1d205cc9b5. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.771934 5687 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.772756 5710 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.772851 5710 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:50:46.772905 5710 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.772950 5710 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 34dd449d08764be182ddfe1d205cc9b5: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:50:46.773383 5693 mvcc.cc:205] Tried to move back new op lower bound from 7315452095571013632 to 7315452095527948288. Current Snapshot: MvccSnapshot[applied={T|T < 7315452095571013632}]
I20260806 06:50:46.773468 5687 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.773546 5687 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 083c80c6b85c4ae087cbaf1613fc3e87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.773584 5688 tablet_replica.cc:439] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34dd449d08764be182ddfe1d205cc9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 } } peers { permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 } } peers { permanent_uuid: "34dd449d08764be182ddfe1d205cc9b5" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37987 } } }
I20260806 06:50:46.773669 5688 tablet_replica.cc:442] T e5704e45415748d088d94177bb8e4be0 P 049fc829e3b240e4bf101b2a22a3864a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:46.777235 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.775058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.3.247.131:37987)
0806 06:50:46.775097 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.775098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.775107 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:46.775169 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.775170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.775181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.775233 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:46.775340 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.775807 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.776534 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.776741 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.776825 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.776984 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.777016 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.777066 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.777132 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.777151 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:46.781997 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.779750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:40009 (local address 127.3.247.131:37987)
0806 06:50:46.779782 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.779784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.779793 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:46.779864 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.779864 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.779873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.779924 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:46.780062 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.780506 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.781255 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.781438 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.781518 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.781689 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.781724 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.781784 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.781868 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.781884 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:46.782001 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.779690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37987 (local address 127.3.247.131:40009)
0806 06:50:46.779747 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:46.779758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:46.779808 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:46.779934 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:46.779935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:46.779936 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:46.779991 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.779992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.780587 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.780588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.781165 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:46.781166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.781519 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.781520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:46.781579 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.781601 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:46.781811 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:46.781812 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:50:46.781864 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:50:46.781904 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 06:50:46.784250 5474 raft_consensus.cc:1261] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Refusing update from remote peer 34dd449d08764be182ddfe1d205cc9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.784238 5334 raft_consensus.cc:1261] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Refusing update from remote peer 34dd449d08764be182ddfe1d205cc9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:46.784458 5710 consensus_queue.cc:1035] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "049fc829e3b240e4bf101b2a22a3864a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.784579 5710 consensus_queue.cc:1035] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "083c80c6b85c4ae087cbaf1613fc3e87" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 38769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:46.795969 5175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:46.796082 5175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:46.796737 5175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4181e68d1ae2480fa6b7741f874057ee: Sending DeleteTablet for 3 replicas of tablet 2d7c5552e27541a283791f0bc550fc7c
I20260806 06:50:46.796988 5314 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c5552e27541a283791f0bc550fc7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:58660
I20260806 06:50:46.797027 5454 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c5552e27541a283791f0bc550fc7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:51116
I20260806 06:50:46.797091 5704 tablet_replica.cc:330] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: stopping tablet replica
I20260806 06:50:46.797140 5704 raft_consensus.cc:2229] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:46.797178 5704 raft_consensus.cc:2258] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.797243 5705 tablet_replica.cc:330] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: stopping tablet replica
I20260806 06:50:46.797302 5705 raft_consensus.cc:2229] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:46.797338 5705 raft_consensus.cc:2258] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.797536 5593 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c5552e27541a283791f0bc550fc7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:46 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:46302
I20260806 06:50:46.797634 5703 tablet_replica.cc:330] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: stopping tablet replica
I20260806 06:50:46.797640 5704 ts_tablet_manager.cc:1905] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.797686 5703 raft_consensus.cc:2229] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:46.797775 5703 raft_consensus.cc:2258] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:46.798096 5705 ts_tablet_manager.cc:1905] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.798108 5703 ts_tablet_manager.cc:1905] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:46.799535 5704 ts_tablet_manager.cc:1918] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:46.799628 5704 log.cc:1196] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/2d7c5552e27541a283791f0bc550fc7c
I20260806 06:50:46.799676 5705 ts_tablet_manager.cc:1918] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:46.799731 5705 log.cc:1196] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/2d7c5552e27541a283791f0bc550fc7c
I20260806 06:50:46.799885 5704 ts_tablet_manager.cc:1939] T 2d7c5552e27541a283791f0bc550fc7c P 049fc829e3b240e4bf101b2a22a3864a: Deleting consensus metadata
I20260806 06:50:46.799965 5705 ts_tablet_manager.cc:1939] T 2d7c5552e27541a283791f0bc550fc7c P 083c80c6b85c4ae087cbaf1613fc3e87: Deleting consensus metadata
I20260806 06:50:46.800182 5162 catalog_manager.cc:4841] TS 049fc829e3b240e4bf101b2a22a3864a (127.3.247.129:43391): tablet 2d7c5552e27541a283791f0bc550fc7c (table test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]) successfully deleted
I20260806 06:50:46.800477 5703 ts_tablet_manager.cc:1918] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:46.800534 5703 log.cc:1196] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/2d7c5552e27541a283791f0bc550fc7c
I20260806 06:50:46.800689 5163 catalog_manager.cc:4841] TS 083c80c6b85c4ae087cbaf1613fc3e87 (127.3.247.130:38769): tablet 2d7c5552e27541a283791f0bc550fc7c (table test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]) successfully deleted
I20260806 06:50:46.800742 5703 ts_tablet_manager.cc:1939] T 2d7c5552e27541a283791f0bc550fc7c P 34dd449d08764be182ddfe1d205cc9b5: Deleting consensus metadata
I20260806 06:50:46.801072 5163 catalog_manager.cc:4841] TS 34dd449d08764be182ddfe1d205cc9b5 (127.3.247.131:37987): tablet 2d7c5552e27541a283791f0bc550fc7c (table test-table [id=55c41afbb11843daa30a0a8ddcb2d5ba]) successfully deleted
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:46.808699 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.799839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.3.247.129:43391)
0806 06:50:46.799883 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.799885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.800179 (+ 294us) server_negotiation.cc:407] Connection header received
0806 06:50:46.800290 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.800291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.800305 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.800364 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.801084 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.801586 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.802367 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.802580 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.802674 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.805843 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.805854 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.805855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.805863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.807474 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.807847 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.807850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.807851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.807916 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.808123 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.808124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.808124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.808281 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.808442 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.808505 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.808589 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:50:46.808974 5366 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58664
I20260806 06:50:46.816458 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:46.809966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.3.247.190:38925)
0806 06:50:46.810016 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:46.810018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:46.810163 (+ 145us) server_negotiation.cc:407] Connection header received
0806 06:50:46.810201 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:46.810202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:46.810215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:46.810287 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:46.810777 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.811584 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.812253 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:46.812583 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:46.812657 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:46.813857 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:46.813858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:46.813858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:46.813866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:46.815388 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.815706 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.815707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.815707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.815771 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:46.815949 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:46.815950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:46.815950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:46.816087 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:46.816202 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:46.816281 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:50:46.816358 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:50:46.816854 5175 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54254
I20260806 06:50:46.817557 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5249
I20260806 06:50:46.824334 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5389
I20260806 06:50:46.830729 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5529
I20260806 06:50:46.835445 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5145
2026-08-06T06:50:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2239 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5739](info): set up 2 sockets
Aug 06 06:50:46 dist-test-slave-0kls krb5kdc[5739](info): commencing operation
krb5kdc: starting...
W20260806 06:50:47.865640 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:47 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:47.985591 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34539
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:48.067934 5752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:48.068154 5752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:48.068176 5752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:48.069269 5752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:48.069303 5752 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:50:48.069315 5752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:48.069329 5752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:48.069340 5752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:48.069352 5752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:48.070900 5752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:34539
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:48.071105 5752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:48.071281 5752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:48.076285 5752 server_base.cc:1034] running on GCE node
W20260806 06:50:48.076205 5760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.076200 5758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.076251 5757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:48.076705 5752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:48.077021 5752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:48.078186 5752 hybrid_clock.cc:648] HybridClock initialized: now 1785999048078181 us; error 37 us; skew 500 ppm
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:48.080718 5752 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:48.081205 5752 webserver.cc:460] Webserver started at http://127.3.247.190:34723/ using document root <none> and password file <none>
I20260806 06:50:48.081440 5752 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:48.081507 5752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:48.081657 5752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:48.082809 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "588a1bca6e3e4e2ea629d1a3ef892d11"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "4986a2b9ce758e75fb8931761a6bf59f"
server_key_iv: "73a4c937f52d7946ed593bbb9f2e265e"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.083205 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "588a1bca6e3e4e2ea629d1a3ef892d11"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "4986a2b9ce758e75fb8931761a6bf59f"
server_key_iv: "73a4c937f52d7946ed593bbb9f2e265e"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.084511 5752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:48.085184 5767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.085350 5752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:48.085453 5752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "588a1bca6e3e4e2ea629d1a3ef892d11"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "4986a2b9ce758e75fb8931761a6bf59f"
server_key_iv: "73a4c937f52d7946ed593bbb9f2e265e"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.085541 5752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:48.108047 5752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:48.110037 5752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:48.110214 5752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:48.114862 5752 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:34539
I20260806 06:50:48.114981 5829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:34539 every 8 connection(s)
I20260806 06:50:48.115302 5752 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:48.115972 5830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.118309 5830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Bootstrap starting.
I20260806 06:50:48.118844 5830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.119072 5830 log.cc:824] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:48.119689 5830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: No bootstrap required, opened a new log
I20260806 06:50:48.120350 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5752
I20260806 06:50:48.120462 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:48.120638 4062 external_mini_cluster.cc:1428] Setting key 63ac8893e45fa45fd1a31b5c3041dfb5
I20260806 06:50:48.121837 5830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } }
I20260806 06:50:48.122015 5830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.122056 5830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588a1bca6e3e4e2ea629d1a3ef892d11, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.122201 5830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } }
I20260806 06:50:48.122285 5830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:48.122308 5830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:48.122386 5830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.123010 5830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } }
I20260806 06:50:48.123112 5830 leader_election.cc:302] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 588a1bca6e3e4e2ea629d1a3ef892d11; no voters:
I20260806 06:50:48.123261 5830 leader_election.cc:288] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:48.123317 5835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:48.123477 5835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [term 1 LEADER]: Becoming Leader. State: Replica: 588a1bca6e3e4e2ea629d1a3ef892d11, State: Running, Role: LEADER
I20260806 06:50:48.123505 5830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:48.123564 5835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } }
I20260806 06:50:48.123870 5836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } } }
I20260806 06:50:48.123966 5836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:48.124130 5837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 588a1bca6e3e4e2ea629d1a3ef892d11. Latest consensus state: current_term: 1 leader_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "588a1bca6e3e4e2ea629d1a3ef892d11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34539 } } }
I20260806 06:50:48.124207 5837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:48.124317 5841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:48.124799 5841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:48.126420 5841 catalog_manager.cc:1347] Generated new cluster ID: 6df6476c3e5644b6b7e5e8e87496e937
I20260806 06:50:48.126474 5841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.132544 5841 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:48.133265 5841 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:48.136060 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.121146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.3.247.190:34539)
0806 06:50:48.121537 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.121548 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.121584 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:50:48.121775 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.121799 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.121868 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.122015 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.122737 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.124178 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.125385 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.126022 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.126137 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.132524 (+ 6387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.132530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.132530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.132546 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.134437 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.134842 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.134846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.134847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.134945 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.135156 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.135156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.135157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.135402 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.135571 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.135680 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.135841 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":83,"threads_started":1}
I20260806 06:50:48.142556 5841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Generated new TSK 0
I20260806 06:50:48.142762 5841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:48.172247 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34539
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:48.287454 5856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:48.287740 5856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:48.287802 5856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:48.289177 5856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:48.289237 5856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:48.289263 5856 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:50:48.289302 5856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:48.289337 5856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:48.290953 5856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:48.291226 5856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:48.291491 5856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:48.297766 5862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.297926 5864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:48.298003 5856 server_base.cc:1034] running on GCE node
W20260806 06:50:48.297757 5861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:48.298327 5856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:48.298614 5856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:48.299803 5856 hybrid_clock.cc:648] HybridClock initialized: now 1785999048299782 us; error 40 us; skew 500 ppm
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:48.302531 5856 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:48.303107 5856 webserver.cc:460] Webserver started at http://127.3.247.129:34049/ using document root <none> and password file <none>
I20260806 06:50:48.303372 5856 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:48.303445 5856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:48.303603 5856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:48.304706 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "a22d73af00954bd6b4160d81833cd91c"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "ab4aa384b342c604cf20671a802ce4b1"
server_key_iv: "32bfea6258ee8e054e749ac900d09d24"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.305080 5856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "a22d73af00954bd6b4160d81833cd91c"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "ab4aa384b342c604cf20671a802ce4b1"
server_key_iv: "32bfea6258ee8e054e749ac900d09d24"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.306632 5856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:48.307576 5871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.307807 5856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:48.307971 5856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "a22d73af00954bd6b4160d81833cd91c"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "ab4aa384b342c604cf20671a802ce4b1"
server_key_iv: "32bfea6258ee8e054e749ac900d09d24"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.308092 5856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:48.331153 5856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:48.333721 5856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:48.333961 5856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:48.334635 5856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:48.334692 5856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.334726 5856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:48.334745 5856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.341830 5856 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:42167
I20260806 06:50:48.341922 5986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:42167 every 8 connection(s)
I20260806 06:50:48.342238 5856 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:48.344866 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.335191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:44085 (local address 127.3.247.190:34539)
0806 06:50:48.335277 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.335280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.335516 (+ 236us) server_negotiation.cc:407] Connection header received
0806 06:50:48.336093 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.336095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.336107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.336165 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.337159 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.338094 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.339229 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.339447 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.339528 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.342103 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.342104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.342105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.342113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.343686 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.344061 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.344061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.344062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.344114 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.344334 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.344334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.344335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.344441 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.344492 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.344727 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.344801 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:50:48.345036 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.335176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.129:44085)
0806 06:50:48.335366 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.335410 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.335973 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.336273 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.336279 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.336715 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.337041 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.337049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.338207 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.338208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.339131 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.339134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.339551 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.339552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.339686 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.340233 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.340242 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.342011 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.343758 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.343763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.343764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.343992 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.344182 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.344182 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.344182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.344278 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.344495 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.344499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.344617 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.344699 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.344826 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260806 06:50:48.345544 5987 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34539
I20260806 06:50:48.345631 5987 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:48.345872 5987 heartbeater.cc:511] Master 127.3.247.190:34539 requested a full tablet report, sending...
I20260806 06:50:48.346407 5784 ts_manager.cc:194] Registered new tserver with Master: a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167)
I20260806 06:50:48.347347 5784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:44085
I20260806 06:50:48.348825 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5856
I20260806 06:50:48.348925 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:48.349124 4062 external_mini_cluster.cc:1428] Setting key 816089ae9968ec2ee50a4d30aa06ce9b
I20260806 06:50:48.352649 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.346736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:46113 (local address 127.3.247.190:34539)
0806 06:50:48.346780 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.346782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.346862 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:50:48.346925 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.346926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.346937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.346980 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.347319 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.347893 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.348682 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.348858 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.348936 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.350043 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.350045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.350045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.350053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.351437 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.351831 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.351832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.351833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.351889 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.352104 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.352105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.352106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.352220 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.352302 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.352461 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.352553 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:50:48.352666 5993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.346652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.129:46113)
0806 06:50:48.346800 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.346811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.346871 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.346986 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.346986 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.347174 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.347243 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.347244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.347979 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.347980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.348589 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.348592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.348949 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.348950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.349032 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.349519 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.349525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.349958 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.351549 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.351550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.351551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.351734 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.351963 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.351964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.351964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.352025 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.352285 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.352286 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.352335 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.352434 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.352554 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:48.378011 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34539
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:48.459666 5996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:48.459875 5996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:48.459903 5996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:48.461071 5996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:48.461113 5996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:48.461135 5996 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:50:48.461193 5996 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:48.461220 5996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:48.462843 5996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:48.463101 5996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:48.463351 5996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:48.468395 6001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.468483 6004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.468402 6002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:48.468585 5996 server_base.cc:1034] running on GCE node
I20260806 06:50:48.468760 5996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:48.468995 5996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:48.470150 5996 hybrid_clock.cc:648] HybridClock initialized: now 1785999048470144 us; error 40 us; skew 500 ppm
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:48.472693 5996 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:48.473136 5996 webserver.cc:460] Webserver started at http://127.3.247.130:45457/ using document root <none> and password file <none>
I20260806 06:50:48.473357 5996 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:48.473426 5996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:48.473578 5996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:48.474764 5996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "3afbef60158d4eb2b648c3c38a85efd0"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "7fd481828673437b73279bb5d5473a34"
server_key_iv: "da8951a7173ef4d8427a3236db92fd53"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.475144 5996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "3afbef60158d4eb2b648c3c38a85efd0"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "7fd481828673437b73279bb5d5473a34"
server_key_iv: "da8951a7173ef4d8427a3236db92fd53"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.476468 5996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:48.477278 6011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.477466 5996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:48.477563 5996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "3afbef60158d4eb2b648c3c38a85efd0"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "7fd481828673437b73279bb5d5473a34"
server_key_iv: "da8951a7173ef4d8427a3236db92fd53"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.477645 5996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:48.486716 5996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:48.489090 5996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:48.489250 5996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:48.489756 5996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:48.489838 5996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.489892 5996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:48.489944 5996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.497141 5996 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:40181
I20260806 06:50:48.497282 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:40181 every 8 connection(s)
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.497552 5996 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:48.500862 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.490353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:38193 (local address 127.3.247.190:34539)
0806 06:50:48.490445 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.490448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.490662 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:50:48.491236 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.491238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.491272 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.491330 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.491988 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.492838 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.494612 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.494926 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.495013 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.498055 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.498057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.498057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.498066 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.499641 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.500050 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.500051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.500051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.500114 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.500346 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.500347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.500347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.500471 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.500530 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.500726 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.500798 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:50:48.501056 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.490338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.130:38193)
0806 06:50:48.490527 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.490570 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.491112 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.491405 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.491409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.491678 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.491873 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.491878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.492968 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.492970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.494504 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.494507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.495047 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.495049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.495208 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.495838 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.495851 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.497925 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.499724 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.499729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.499731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.499958 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.500183 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.500183 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.500183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.500279 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.500522 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.500525 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.500634 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.500713 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.500832 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260806 06:50:48.501528 6127 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34539
I20260806 06:50:48.501617 6127 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:48.501896 6127 heartbeater.cc:511] Master 127.3.247.190:34539 requested a full tablet report, sending...
I20260806 06:50:48.502279 5783 ts_manager.cc:194] Registered new tserver with Master: 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:48.502831 5783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:38193
I20260806 06:50:48.502945 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 5996
I20260806 06:50:48.503026 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:48.503244 4062 external_mini_cluster.cc:1428] Setting key 55feaba8ac596951590db19fff6d101e
I20260806 06:50:48.509321 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.502642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.130:57419)
0806 06:50:48.502774 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.502788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.502844 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.503084 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.503085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.503296 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.503368 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.503370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.504249 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.504249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.504748 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.504749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.505244 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.505245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.505335 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.505755 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.505761 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.506261 (+ 500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.508225 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.508226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.508226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.508415 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.508636 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.508636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.508637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.508691 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.509070 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.509070 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.509109 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.509160 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.509246 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260806 06:50:48.509418 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.502715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:57419 (local address 127.3.247.190:34539)
0806 06:50:48.502968 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.502970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.502978 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:48.502997 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.502998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.503008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.503064 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.503482 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.504155 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.504827 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.505143 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.505217 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.506377 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.506378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.506379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.506387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.508143 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.508494 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.508496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.508497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.508567 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.508757 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.508758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.508759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.508997 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.509080 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.509193 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.509280 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:48.529945 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34539
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:48.611786 6136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:48.612038 6136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:48.612079 6136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:48.613293 6136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:48.613363 6136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:48.613396 6136 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:50:48.613426 6136 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:50:48.613471 6136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:48.615056 6136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:48.615350 6136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:48.615605 6136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:48.620829 6144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.620869 6142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:48.620814 6141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:48.620885 6136 server_base.cc:1034] running on GCE node
I20260806 06:50:48.621191 6136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:48.621387 6136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:48.622560 6136 hybrid_clock.cc:648] HybridClock initialized: now 1785999048622548 us; error 26 us; skew 500 ppm
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:48.625207 6136 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:48.625660 6136 webserver.cc:460] Webserver started at http://127.3.247.131:45651/ using document root <none> and password file <none>
I20260806 06:50:48.625922 6136 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:48.625972 6136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:48.626060 6136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:48.627025 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "a48694399d75472fa8386e1bdc03d8f1"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "3261d0b262f954383fa06c7f12970a31"
server_key_iv: "1c498a3373d4415e57bbf9e81241f6e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.627307 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "a48694399d75472fa8386e1bdc03d8f1"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "3261d0b262f954383fa06c7f12970a31"
server_key_iv: "1c498a3373d4415e57bbf9e81241f6e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.628494 6136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:50:48.629132 6151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.629282 6136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:48.629361 6136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "a48694399d75472fa8386e1bdc03d8f1"
format_stamp: "Formatted at 2026-08-06 06:50:48 on dist-test-slave-0kls"
server_key: "3261d0b262f954383fa06c7f12970a31"
server_key_iv: "1c498a3373d4415e57bbf9e81241f6e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:48.629420 6136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:48.642195 6136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:48.644245 6136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:48.644356 6136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:48.644876 6136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:48.644914 6136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.645018 6136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:48.645059 6136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:48.651571 6136 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:38387
I20260806 06:50:48.651633 6266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:38387 every 8 connection(s)
I20260806 06:50:48.651976 6136 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.654696 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6136
I20260806 06:50:48.654783 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:48.654989 4062 external_mini_cluster.cc:1428] Setting key 184bfa9848d37e12158a465538bd201b
I20260806 06:50:48.655159 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.645443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:35309 (local address 127.3.247.190:34539)
0806 06:50:48.645536 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.645539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.645812 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:50:48.646378 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.646379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.646394 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.646482 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.647305 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.648150 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.649686 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.650017 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.650106 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.652664 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.652665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.652665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.652672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.653880 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.654241 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.654241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.654242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.654293 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.654476 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.654477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.654477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.654575 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.654626 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.654970 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.655057 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:50:48.655385 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.645405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.131:35309)
0806 06:50:48.645641 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.645670 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.646277 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.646538 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.646541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.646899 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.647198 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.647204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.648245 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.648246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.649589 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.649592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.650107 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.650108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.650212 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.650828 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.650839 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.652580 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.653957 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.653962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.653964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.654175 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.654333 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.654334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.654334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.654421 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.654646 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.654652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.654821 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.654953 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.655127 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260806 06:50:48.656752 6267 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34539
I20260806 06:50:48.656864 6267 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:48.657043 6267 heartbeater.cc:511] Master 127.3.247.190:34539 requested a full tablet report, sending...
I20260806 06:50:48.657435 5782 ts_manager.cc:194] Registered new tserver with Master: a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387)
I20260806 06:50:48.657907 5782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:35309
I20260806 06:50:48.662531 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.656709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:46843 (local address 127.3.247.190:34539)
0806 06:50:48.656746 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.656748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.657321 (+ 573us) server_negotiation.cc:407] Connection header received
0806 06:50:48.657396 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.657397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.657408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.657470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.657840 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.658556 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.659263 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.659441 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.659495 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.660291 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.660292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.660293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.660299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.661650 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.661996 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.661997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.661997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.662044 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.662188 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.662189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.662189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.662271 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.662315 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.662423 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.662475 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:48.662533 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.656664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.3.247.131:46843)
0806 06:50:48.657265 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.657277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.657333 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.657472 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.657472 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.657665 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.657714 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.657715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.658649 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.658649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.659191 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.659193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.659504 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.659505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.659557 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.659938 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.659942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.660219 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.661715 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.661716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.661716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.661931 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.662099 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.662100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.662100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.662149 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.662318 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.662318 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.662353 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.662401 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.662472 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":538,"threads_started":1}
I20260806 06:50:48.667466 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.689524 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.682531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.3.247.190:34539)
0806 06:50:48.682598 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.682600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.682609 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:48.682635 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.682636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.682653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.682716 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.683290 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.683782 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.684589 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.684776 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.684878 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.687030 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.687031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.687032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.687039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.688412 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.688754 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.688755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.688756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.688814 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.689016 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.689017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.689017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.689129 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.689235 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.689335 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.689420 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:50:48.691010 5782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:48.692493 5782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:48.699296 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.694783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40181 (local address 127.0.0.1:55412)
0806 06:50:48.694954 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.694983 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.695062 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.695468 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.695471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.695473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:48.695538 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.695541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.696877 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.696878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.698273 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.698275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.698977 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.698978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.699056 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.699069 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.699139 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.699192 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":41,"thread_start_us":90,"threads_started":1}
I20260806 06:50:48.699455 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.694970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.3.247.129:42167)
0806 06:50:48.695335 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.695340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.695355 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:50:48.695386 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.695388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.695427 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.695502 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:48.695703 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.696788 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.698346 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.699062 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.699135 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.699185 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.699286 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.699320 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":71,"threads_started":1}
I20260806 06:50:48.699700 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.695137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.3.247.130:40181)
0806 06:50:48.695308 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.695313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.695322 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:48.695360 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.695362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.695377 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.695453 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:48.695606 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.696775 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.698376 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.698885 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.698977 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.699036 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.699480 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.699607 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260806 06:50:48.699653 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.694785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42167 (local address 127.0.0.1:45748)
0806 06:50:48.695169 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.695178 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.695246 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.695565 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.695566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.695568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:48.695621 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.695623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.696879 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.696880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.698244 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.698247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.699150 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.699151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.699213 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.699216 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.699274 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.699304 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":41,"thread_start_us":57,"threads_started":1}
I20260806 06:50:48.700551 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.694796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:38387 (local address 127.0.0.1:52472)
0806 06:50:48.696913 (+ 2117us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.696923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.697001 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.697175 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.697176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.697178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:48.697216 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.697218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.698225 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.698227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.699328 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.699330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.700338 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.700339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.700397 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.700401 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.700447 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.700482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2098,"thread_start_us":90,"threads_started":1}
I20260806 06:50:48.700601 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.695076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52472 (local address 127.3.247.131:38387)
0806 06:50:48.695259 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.695265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.696995 (+ 1730us) server_negotiation.cc:407] Connection header received
0806 06:50:48.697066 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.697068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.697082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.697168 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:48.697268 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.698148 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.699429 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.700225 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.700313 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.700362 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.700469 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.700505 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260806 06:50:48.700587 6061 tablet_service.cc:1467] Processing CreateTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe (DEFAULT_TABLE table=test-table [id=500029ff74054db58f1a1ff3f6fd5b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.700618 5921 tablet_service.cc:1467] Processing CreateTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe (DEFAULT_TABLE table=test-table [id=500029ff74054db58f1a1ff3f6fd5b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.700971 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df1b89b26cd4aa099fdd013d86edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.700982 6061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df1b89b26cd4aa099fdd013d86edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.703886 6290 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Bootstrap starting.
I20260806 06:50:48.703934 6291 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Bootstrap starting.
I20260806 06:50:48.704360 6201 tablet_service.cc:1467] Processing CreateTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe (DEFAULT_TABLE table=test-table [id=500029ff74054db58f1a1ff3f6fd5b56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.704502 6291 tablet_bootstrap.cc:654] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.704516 6290 tablet_bootstrap.cc:654] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.704743 6201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df1b89b26cd4aa099fdd013d86edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.704911 6291 log.cc:824] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:48.704913 6290 log.cc:824] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:48.705590 6291 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: No bootstrap required, opened a new log
I20260806 06:50:48.705673 6291 ts_tablet_manager.cc:1397] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:48.706830 6290 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: No bootstrap required, opened a new log
I20260806 06:50:48.706899 6290 ts_tablet_manager.cc:1397] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:50:48.707424 6291 raft_consensus.cc:348] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.707597 6291 raft_consensus.cc:374] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.707641 6291 raft_consensus.cc:729] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afbef60158d4eb2b648c3c38a85efd0, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.707752 6291 consensus_queue.cc:260] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.707789 6294 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Bootstrap starting.
I20260806 06:50:48.708029 6291 ts_tablet_manager.cc:1428] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:50:48.708063 6127 heartbeater.cc:503] Master 127.3.247.190:34539 was elected leader, sending a full tablet report...
I20260806 06:50:48.708679 6294 tablet_bootstrap.cc:654] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.709034 6294 log.cc:824] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:48.709064 6290 raft_consensus.cc:348] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.709219 6290 raft_consensus.cc:374] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.709259 6290 raft_consensus.cc:729] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.709376 6290 consensus_queue.cc:260] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.709640 6290 ts_tablet_manager.cc:1428] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:50:48.709748 5987 heartbeater.cc:503] Master 127.3.247.190:34539 was elected leader, sending a full tablet report...
I20260806 06:50:48.709975 6294 tablet_bootstrap.cc:492] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: No bootstrap required, opened a new log
I20260806 06:50:48.710044 6294 ts_tablet_manager.cc:1397] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:48.711580 6294 raft_consensus.cc:348] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.711692 6294 raft_consensus.cc:374] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.711712 6294 raft_consensus.cc:729] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.711822 6294 consensus_queue.cc:260] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.712076 6294 ts_tablet_manager.cc:1428] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:48.712126 6267 heartbeater.cc:503] Master 127.3.247.190:34539 was elected leader, sending a full tablet report...
W20260806 06:50:48.748214 6128 tablet.cc:2369] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:48.772543 6298 raft_consensus.cc:482] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:48.772661 6298 raft_consensus.cc:504] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.772946 6298 leader_election.cc:288] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:48.776192 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.773262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:34731 (local address 127.3.247.129:42167)
0806 06:50:48.773302 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.773304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.773311 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:48.773361 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.773362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.773372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.773455 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:48.773650 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.774206 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775408 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.775778 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775831 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.775837 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.776073 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.776115 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:48.776226 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.773148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42167 (local address 127.3.247.131:34731)
0806 06:50:48.773206 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.773219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.773263 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.773489 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.773491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.773493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:48.773569 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.773572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.774325 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.774326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.775304 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.775307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.775875 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.775974 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.775979 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.776063 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.776106 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:50:48.776316 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.773228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:42495 (local address 127.3.247.130:40181)
0806 06:50:48.773276 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.773278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.773496 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:50:48.773528 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.773529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.773540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.773601 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:48.773853 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.774510 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775446 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.775901 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775986 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.775995 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.776184 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.776230 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:50:48.776399 6299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.773149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40181 (local address 127.3.247.131:42495)
0806 06:50:48.773412 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.773423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.773483 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.773635 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.773636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.773639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:48.773705 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.773707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.774609 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.774610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.775369 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.775372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.775990 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.775990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.776095 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.776099 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.776154 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.776189 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":61,"thread_start_us":133,"threads_started":1}
I20260806 06:50:48.776743 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df1b89b26cd4aa099fdd013d86edafe" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c" is_pre_election: true
I20260806 06:50:48.776801 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df1b89b26cd4aa099fdd013d86edafe" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0" is_pre_election: true
I20260806 06:50:48.776870 5941 raft_consensus.cc:2454] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:48.776997 6081 raft_consensus.cc:2454] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:48.777153 6155 leader_election.cc:302] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:48.777289 6298 raft_consensus.cc:2781] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:48.777369 6298 raft_consensus.cc:482] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:48.777410 6298 raft_consensus.cc:3037] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.778115 6298 raft_consensus.cc:504] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.778254 6298 leader_election.cc:288] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Requested vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:48.778434 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df1b89b26cd4aa099fdd013d86edafe" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c"
I20260806 06:50:48.778492 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df1b89b26cd4aa099fdd013d86edafe" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0"
I20260806 06:50:48.778532 5941 raft_consensus.cc:3037] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.778584 6081 raft_consensus.cc:3037] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.779201 5941 raft_consensus.cc:2454] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:48.779392 6081 raft_consensus.cc:2454] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:48.779423 6155 leader_election.cc:302] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:48.779592 6298 raft_consensus.cc:2781] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:48.779773 6298 raft_consensus.cc:686] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Becoming Leader. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Running, Role: LEADER
I20260806 06:50:48.779865 6298 consensus_queue.cc:237] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.780689 5783 catalog_manager.cc:5495] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } health_report { overall_health: HEALTHY } } }
Aug 06 06:50:48 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999048, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:48.813442 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.806449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34539 (local address 127.0.0.1:55732)
0806 06:50:48.806611 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.806621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.806685 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.806820 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.806821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.807136 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:48.807184 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.807186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.807822 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.807823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.808357 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.808358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.808688 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.808689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.808727 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.809219 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:48.809223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:48.810664 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:48.812328 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.812335 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.812337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.812537 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.812712 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:48.812713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:48.812713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:48.812769 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:48.813167 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:48.813173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:48.813212 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.813259 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.813346 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260806 06:50:48.813539 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.806522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.3.247.190:34539)
0806 06:50:48.806626 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.806630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.806686 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:50:48.806738 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.806739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.806748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.806804 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:48.807257 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.807747 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.808436 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.808611 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.808677 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.810778 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:48.810779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:48.810780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:48.810786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:48.812262 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.812609 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.812610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.812610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.812658 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:48.812982 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:48.812983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:48.812983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:48.813104 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:48.813168 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.813284 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.813395 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":76}
I20260806 06:50:48.815567 5783 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55732:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:50:48.815763 5783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:48.817240 6201 tablet_service.cc:1467] Processing CreateTablet for tablet 09cabe7015354fb9b35e9073aa9e9f02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700e6edeb6674d8f860844be9c62d1cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:48.817225 6061 tablet_service.cc:1467] Processing CreateTablet for tablet 09cabe7015354fb9b35e9073aa9e9f02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700e6edeb6674d8f860844be9c62d1cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:48.817299 5921 tablet_service.cc:1467] Processing CreateTablet for tablet 09cabe7015354fb9b35e9073aa9e9f02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=700e6edeb6674d8f860844be9c62d1cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:50:48.817411 6061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cabe7015354fb9b35e9073aa9e9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.817411 6201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cabe7015354fb9b35e9073aa9e9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.817509 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cabe7015354fb9b35e9073aa9e9f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.818746 6290 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: Bootstrap starting.
I20260806 06:50:48.819051 6291 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: Bootstrap starting.
I20260806 06:50:48.819084 6294 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: Bootstrap starting.
I20260806 06:50:48.819484 6290 tablet_bootstrap.cc:654] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.819516 6294 tablet_bootstrap.cc:654] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.819648 6291 tablet_bootstrap.cc:654] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.820222 6290 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: No bootstrap required, opened a new log
I20260806 06:50:48.820279 6290 ts_tablet_manager.cc:1397] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:48.820330 6291 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: No bootstrap required, opened a new log
I20260806 06:50:48.820374 6294 tablet_bootstrap.cc:492] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: No bootstrap required, opened a new log
I20260806 06:50:48.820382 6291 ts_tablet_manager.cc:1397] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:48.820420 6294 ts_tablet_manager.cc:1397] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:48.820447 6290 raft_consensus.cc:348] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820529 6290 raft_consensus.cc:374] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.820549 6290 raft_consensus.cc:729] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.820552 6294 raft_consensus.cc:348] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820554 6291 raft_consensus.cc:348] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820617 6290 consensus_queue.cc:260] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820649 6294 raft_consensus.cc:374] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.820652 6291 raft_consensus.cc:374] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.820674 6294 raft_consensus.cc:729] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.820677 6291 raft_consensus.cc:729] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afbef60158d4eb2b648c3c38a85efd0, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.820715 6290 ts_tablet_manager.cc:1428] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:48.820715 6294 consensus_queue.cc:260] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820725 6291 consensus_queue.cc:260] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.820797 6294 ts_tablet_manager.cc:1428] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:48.820816 6291 ts_tablet_manager.cc:1428] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:48.820859 6297 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.820950 6297 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.820979 6295 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.821015 6298 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.821058 6295 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.821105 6298 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.822681 6298 raft_consensus.cc:482] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:48.822742 6298 raft_consensus.cc:504] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.822873 6298 leader_election.cc:288] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:48.823026 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cabe7015354fb9b35e9073aa9e9f02" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c" is_pre_election: true
I20260806 06:50:48.823050 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cabe7015354fb9b35e9073aa9e9f02" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0" is_pre_election: true
I20260806 06:50:48.823098 5941 raft_consensus.cc:2454] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:48.823109 6081 raft_consensus.cc:2454] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:48.823216 6155 leader_election.cc:302] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:48.823312 6298 raft_consensus.cc:2781] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:48.823349 6298 raft_consensus.cc:482] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:48.823376 6298 raft_consensus.cc:3037] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.823988 6298 raft_consensus.cc:504] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.824115 6298 leader_election.cc:288] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Requested vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:48.824281 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cabe7015354fb9b35e9073aa9e9f02" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c"
I20260806 06:50:48.824297 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cabe7015354fb9b35e9073aa9e9f02" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0"
I20260806 06:50:48.824347 5941 raft_consensus.cc:3037] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.824362 6081 raft_consensus.cc:3037] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:48.824918 5941 raft_consensus.cc:2454] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:48.825024 6081 raft_consensus.cc:2454] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:48.825071 6155 leader_election.cc:302] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:48.825162 6298 raft_consensus.cc:2781] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:48.825209 6298 raft_consensus.cc:686] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Becoming Leader. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Running, Role: LEADER
I20260806 06:50:48.825256 6298 consensus_queue.cc:237] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.825578 6300 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: TxnStatusTablet state changed. Reason: New leader a48694399d75472fa8386e1bdc03d8f1. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.825675 6300 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:50:48.825882 5782 catalog_manager.cc:5495] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } health_report { overall_health: HEALTHY } } }
I20260806 06:50:48.825915 6312 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:50:48.826068 6312 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:50:48.831769 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.828991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:38387 (local address 127.0.0.1:52488)
0806 06:50:48.829067 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.829077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.829146 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.829347 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.829347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.829348 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:48.829398 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.829400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.830198 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.830200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.830769 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.830771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.831160 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.831161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.831228 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.831248 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:48.831599 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:48.831605 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.831658 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.831704 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:50:48.831785 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.829065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.3.247.131:38387)
0806 06:50:48.829123 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.829125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.829134 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:48.829226 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.829227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.829257 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.829338 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:48.829472 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.830117 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.830867 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.831076 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.831159 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.831374 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:48.831518 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:48.831630 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.831661 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.831691 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:50:48.833490 6081 raft_consensus.cc:1261] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:48.833823 5941 raft_consensus.cc:1261] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:48.833993 6300 consensus_queue.cc:1035] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:48.834090 6298 consensus_queue.cc:1035] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:48.835318 6297 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: TxnStatusTablet state changed. Reason: New leader a48694399d75472fa8386e1bdc03d8f1. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.835429 6297 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.835507 6295 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: TxnStatusTablet state changed. Reason: New leader a48694399d75472fa8386e1bdc03d8f1. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.835628 6295 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.836570 6300 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.836655 6300 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:50:48.836820 6316 mvcc.cc:205] Tried to move back new op lower bound from 7315452104019783680 to 7315452103988662272. Current Snapshot: MvccSnapshot[applied={T|T < 7315452104019783680}]
I20260806 06:50:48.837126 6295 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.837211 6295 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P 3afbef60158d4eb2b648c3c38a85efd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.837738 6315 mvcc.cc:205] Tried to move back new op lower bound from 7315452104019783680 to 7315452103988662272. Current Snapshot: MvccSnapshot[applied={T|T < 7315452104019783680 or (T in {7315452104019783680})}]
I20260806 06:50:48.841081 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.838593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.3.247.131:38387)
0806 06:50:48.838643 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.838645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.838654 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:48.838713 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.838714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.838723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.838783 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:48.838904 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.839643 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.840347 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.840658 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.840726 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.840855 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:48.840887 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:48.840938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.840991 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.841008 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:48.842563 6298 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.842654 6298 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a48694399d75472fa8386e1bdc03d8f1: This TxnStatusTablet replica's current role is: LEADER
W20260806 06:50:48.842959 5988 tablet.cc:2369] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:48.843127 6317 mvcc.cc:205] Tried to move back new op lower bound from 7315452104019783680 to 7315452103988662272. Current Snapshot: MvccSnapshot[applied={T|T < 7315452104019783680}]
I20260806 06:50:48.843180 6297 tablet_replica.cc:439] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } } }
I20260806 06:50:48.843261 6297 tablet_replica.cc:442] T 09cabe7015354fb9b35e9073aa9e9f02 P a22d73af00954bd6b4160d81833cd91c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:50:48.847353 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.843938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:38387 (local address 127.3.247.131:36641)
0806 06:50:48.844014 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:48.844025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:48.844063 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:48.844214 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:48.844215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:48.844216 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:48.844262 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.844263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.845201 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.845202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.846445 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:48.846447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.846943 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.846944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:48.846997 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.847011 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:48.847203 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:48.847204 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:50:48.847239 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 06:50:48.847272 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 06:50:48.847353 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:48.843977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36641 (local address 127.3.247.131:38387)
0806 06:50:48.844121 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:48.844123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:48.844128 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:50:48.844143 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:48.844144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:48.844151 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:48.844197 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:48.844339 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.845097 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.846538 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:48.846834 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:48.846908 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:48.847103 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:48.847140 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:48.847197 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:48.847242 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:50:48.847260 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 06:50:48.849648 5941 raft_consensus.cc:1261] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:48.849648 6081 raft_consensus.cc:1261] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:48.849881 6324 consensus_queue.cc:1035] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:48.849997 6298 consensus_queue.cc:1035] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:48.864192 5783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:48.864336 5783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:48.865185 5783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Sending DeleteTablet for 3 replicas of tablet 8df1b89b26cd4aa099fdd013d86edafe
I20260806 06:50:48.865582 5921 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:48 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:45748
I20260806 06:50:48.865618 6061 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:48 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55412
I20260806 06:50:48.865599 6200 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df1b89b26cd4aa099fdd013d86edafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:48 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:52472
I20260806 06:50:48.865921 6333 tablet_replica.cc:330] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: stopping tablet replica
I20260806 06:50:48.866034 6333 raft_consensus.cc:2229] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:48.866130 6333 raft_consensus.cc:2258] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:48.866271 6331 tablet_replica.cc:330] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: stopping tablet replica
I20260806 06:50:48.866282 6332 tablet_replica.cc:330] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: stopping tablet replica
I20260806 06:50:48.866269 5783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:50:48.866391 6332 raft_consensus.cc:2229] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:48.866458 6332 raft_consensus.cc:2258] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:50:48.866451 5783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:48.866616 6333 ts_tablet_manager.cc:1905] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:48.866724 6331 raft_consensus.cc:2229] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:48.867110 6332 ts_tablet_manager.cc:1905] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:48.867164 6331 raft_consensus.cc:2258] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:48.867867 6200 tablet_service.cc:1467] Processing CreateTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (DEFAULT_TABLE table=test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.868023 6200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.868057 6331 ts_tablet_manager.cc:1905] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:48.868726 6333 ts_tablet_manager.cc:1918] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:48.868804 6333 log.cc:1196] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/8df1b89b26cd4aa099fdd013d86edafe
I20260806 06:50:48.869099 6333 ts_tablet_manager.cc:1939] T 8df1b89b26cd4aa099fdd013d86edafe P a48694399d75472fa8386e1bdc03d8f1: Deleting consensus metadata
I20260806 06:50:48.869493 5769 catalog_manager.cc:4841] TS a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387): tablet 8df1b89b26cd4aa099fdd013d86edafe (table test-table [id=500029ff74054db58f1a1ff3f6fd5b56]) successfully deleted
I20260806 06:50:48.869540 6294 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Bootstrap starting.
I20260806 06:50:48.869688 6332 ts_tablet_manager.cc:1918] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:48.869822 6332 log.cc:1196] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/8df1b89b26cd4aa099fdd013d86edafe
I20260806 06:50:48.870018 6294 tablet_bootstrap.cc:654] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.870054 6331 ts_tablet_manager.cc:1918] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:48.870124 6331 log.cc:1196] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/8df1b89b26cd4aa099fdd013d86edafe
I20260806 06:50:48.870131 6332 ts_tablet_manager.cc:1939] T 8df1b89b26cd4aa099fdd013d86edafe P 3afbef60158d4eb2b648c3c38a85efd0: Deleting consensus metadata
I20260806 06:50:48.870468 6331 ts_tablet_manager.cc:1939] T 8df1b89b26cd4aa099fdd013d86edafe P a22d73af00954bd6b4160d81833cd91c: Deleting consensus metadata
I20260806 06:50:48.870612 5768 catalog_manager.cc:4841] TS 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181): tablet 8df1b89b26cd4aa099fdd013d86edafe (table test-table [id=500029ff74054db58f1a1ff3f6fd5b56]) successfully deleted
I20260806 06:50:48.870683 6061 tablet_service.cc:1467] Processing CreateTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (DEFAULT_TABLE table=test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.870786 6061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.871003 5771 catalog_manager.cc:4841] TS a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167): tablet 8df1b89b26cd4aa099fdd013d86edafe (table test-table [id=500029ff74054db58f1a1ff3f6fd5b56]) successfully deleted
I20260806 06:50:48.871608 6294 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: No bootstrap required, opened a new log
I20260806 06:50:48.871646 6294 ts_tablet_manager.cc:1397] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:48.871778 6294 raft_consensus.cc:348] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.871832 6294 raft_consensus.cc:374] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.871850 6294 raft_consensus.cc:729] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.871943 6291 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Bootstrap starting.
I20260806 06:50:48.871922 6294 consensus_queue.cc:260] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.871946 5921 tablet_service.cc:1467] Processing CreateTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (DEFAULT_TABLE table=test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:48.872040 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:48.872030 6294 ts_tablet_manager.cc:1428] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:48.872361 6291 tablet_bootstrap.cc:654] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.873188 6290 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Bootstrap starting.
I20260806 06:50:48.873549 6291 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: No bootstrap required, opened a new log
I20260806 06:50:48.873603 6291 ts_tablet_manager.cc:1397] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:48.873665 6290 tablet_bootstrap.cc:654] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:48.873869 6291 raft_consensus.cc:348] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.873972 6291 raft_consensus.cc:374] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.873998 6291 raft_consensus.cc:729] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afbef60158d4eb2b648c3c38a85efd0, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.874055 6291 consensus_queue.cc:260] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.874162 6291 ts_tablet_manager.cc:1428] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:50:48.874636 6290 tablet_bootstrap.cc:492] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: No bootstrap required, opened a new log
I20260806 06:50:48.874689 6290 ts_tablet_manager.cc:1397] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:50:48.874833 6290 raft_consensus.cc:348] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.874900 6290 raft_consensus.cc:374] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:48.874924 6290 raft_consensus.cc:729] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Initialized, Role: FOLLOWER
I20260806 06:50:48.874969 6290 consensus_queue.cc:260] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:48.875072 6290 ts_tablet_manager.cc:1428] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 06:50:48.902525 6268 tablet.cc:2369] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:49.130985 6297 raft_consensus.cc:482] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:49.131083 6297 raft_consensus.cc:504] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:49.131532 6297 leader_election.cc:288] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181), a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387)
I20260806 06:50:49.134697 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.131804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:43953 (local address 127.3.247.130:40181)
0806 06:50:49.131859 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:49.131861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:49.131920 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:50:49.132049 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:49.132051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:49.132060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:49.132120 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:49.132265 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.132896 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.133999 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.134343 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.134398 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.134432 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:49.134580 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:50:49.134628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:50:49.134730 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.131734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40181 (local address 127.3.247.129:43953)
0806 06:50:49.131840 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:49.131853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:49.131931 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:49.132132 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:49.132133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:49.132135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:49.132207 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.132209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.133026 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.133027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.133901 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.133903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.134430 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.134430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.134511 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.134514 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:49.134556 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:50:49.134589 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":40,"thread_start_us":63,"threads_started":1}
I20260806 06:50:49.134755 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.131772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:38387 (local address 127.3.247.129:36731)
0806 06:50:49.131919 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:49.131930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:49.131999 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:49.132194 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:49.132195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:49.132196 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:49.132265 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.132266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.132975 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.132976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.133882 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.133885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.134462 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.134463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.134561 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.134566 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:50:49.134622 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:50:49.134656 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260806 06:50:49.134760 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.131860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:36731 (local address 127.3.247.131:38387)
0806 06:50:49.131915 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:49.131917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:49.132000 (+ 83us) server_negotiation.cc:407] Connection header received
0806 06:50:49.132106 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:49.132108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:49.132118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:49.132170 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:49.132340 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.132900 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.133972 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.134348 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.134438 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.134448 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:49.134642 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:50:49.134685 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:49.135051 6221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a13c9aaf9048aa9692ca5bf23ad2a4" candidate_uuid: "a22d73af00954bd6b4160d81833cd91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48694399d75472fa8386e1bdc03d8f1" is_pre_election: true
I20260806 06:50:49.135079 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a13c9aaf9048aa9692ca5bf23ad2a4" candidate_uuid: "a22d73af00954bd6b4160d81833cd91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0" is_pre_election: true
I20260806 06:50:49.135166 6081 raft_consensus.cc:2454] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a22d73af00954bd6b4160d81833cd91c in term 0.
I20260806 06:50:49.135208 6221 raft_consensus.cc:2454] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a22d73af00954bd6b4160d81833cd91c in term 0.
I20260806 06:50:49.135342 5872 leader_election.cc:302] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afbef60158d4eb2b648c3c38a85efd0, a22d73af00954bd6b4160d81833cd91c; no voters:
I20260806 06:50:49.135495 6297 raft_consensus.cc:2781] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:49.135535 6297 raft_consensus.cc:482] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:49.135550 6297 raft_consensus.cc:3037] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:49.136214 6297 raft_consensus.cc:504] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:49.136353 6297 leader_election.cc:288] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [CANDIDATE]: Term 1 election: Requested vote from peers 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181), a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387)
I20260806 06:50:49.136533 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a13c9aaf9048aa9692ca5bf23ad2a4" candidate_uuid: "a22d73af00954bd6b4160d81833cd91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0"
I20260806 06:50:49.136533 6221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82a13c9aaf9048aa9692ca5bf23ad2a4" candidate_uuid: "a22d73af00954bd6b4160d81833cd91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48694399d75472fa8386e1bdc03d8f1"
I20260806 06:50:49.136600 6081 raft_consensus.cc:3037] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:49.136600 6221 raft_consensus.cc:3037] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:49.137321 6081 raft_consensus.cc:2454] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a22d73af00954bd6b4160d81833cd91c in term 1.
I20260806 06:50:49.137388 6221 raft_consensus.cc:2454] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a22d73af00954bd6b4160d81833cd91c in term 1.
I20260806 06:50:49.137511 5872 leader_election.cc:302] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afbef60158d4eb2b648c3c38a85efd0, a22d73af00954bd6b4160d81833cd91c; no voters:
I20260806 06:50:49.137586 6297 raft_consensus.cc:2781] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:49.137722 6297 raft_consensus.cc:686] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 1 LEADER]: Becoming Leader. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Running, Role: LEADER
I20260806 06:50:49.137774 6297 consensus_queue.cc:237] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:49.138476 5783 catalog_manager.cc:5495] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c reported cstate change: term changed from 0 to 1, leader changed from <none> to a22d73af00954bd6b4160d81833cd91c (127.3.247.129). New cstate: current_term: 1 leader_uuid: "a22d73af00954bd6b4160d81833cd91c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } health_report { overall_health: UNKNOWN } } }
I20260806 06:50:49.159629 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.157576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.3.247.129:42167)
0806 06:50:49.157612 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:49.157614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:49.157622 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:49.157677 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:49.157678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:49.157695 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:49.157802 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:49.157906 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.158362 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.159030 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.159206 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.159255 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.159364 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:49.159473 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:49.159522 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:49.159560 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:50:49.159574 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:49.164067 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.162027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40967 (local address 127.3.247.131:38387)
0806 06:50:49.162098 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:49.162100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:49.162170 (+ 70us) server_negotiation.cc:407] Connection header received
0806 06:50:49.162226 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:49.162226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:49.162236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:49.162274 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:49.162394 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.162822 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.163521 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.163694 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.163755 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.163886 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:49.163909 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:49.163951 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:49.163999 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:50:49.164013 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:50:49.164131 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.161957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:38387 (local address 127.3.247.129:40967)
0806 06:50:49.162116 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:49.162125 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:49.162177 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:49.162288 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:49.162288 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:49.162290 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:49.162338 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.162339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.162879 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.162879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.163450 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.163452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.163768 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.163768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.163809 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.163821 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:49.163966 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:49.163967 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:50:49.163997 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:50:49.164028 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260806 06:50:49.166145 6081 raft_consensus.cc:1261] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Refusing update from remote peer a22d73af00954bd6b4160d81833cd91c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:49.166214 6221 raft_consensus.cc:1261] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Refusing update from remote peer a22d73af00954bd6b4160d81833cd91c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:49.166424 6339 consensus_queue.cc:1035] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:49.166599 6297 consensus_queue.cc:1035] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:49.173676 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.171354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36141 (local address 127.3.247.129:42167)
0806 06:50:49.171480 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:49.171481 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:49.171488 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:49.171509 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:49.171510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:49.171519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:49.171569 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:49.171680 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.172184 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.172879 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.173171 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.173234 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.173435 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:49.173479 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:49.173525 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:49.173586 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:50:49.173608 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 06:50:49.173708 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:49.171309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42167 (local address 127.3.247.131:36141)
0806 06:50:49.171354 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:49.171365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:49.171400 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:49.171576 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:49.171577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:49.171578 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:50:49.171623 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.171625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.172251 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.172252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.172783 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:49.172785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:49.173278 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:49.173279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:49.173362 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:49.173368 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:50:49.173539 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:50:49.173541 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:50:49.173585 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:50:49.173635 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:50:49.174041 6317 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:50:50.692992 5782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:50.693091 5782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:50.693895 5782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Sending DeleteTablet for 3 replicas of tablet 82a13c9aaf9048aa9692ca5bf23ad2a4
I20260806 06:50:50.694175 6200 tablet_service.cc:1514] Processing DeleteTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:52472
I20260806 06:50:50.694281 6061 tablet_service.cc:1514] Processing DeleteTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55412
I20260806 06:50:50.694397 5921 tablet_service.cc:1514] Processing DeleteTablet for tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:45748
I20260806 06:50:50.694459 6370 tablet_replica.cc:330] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: stopping tablet replica
I20260806 06:50:50.694510 6370 raft_consensus.cc:2229] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:50.694547 6370 raft_consensus.cc:2258] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.694571 6371 tablet_replica.cc:330] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: stopping tablet replica
I20260806 06:50:50.694727 6369 tablet_replica.cc:330] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: stopping tablet replica
I20260806 06:50:50.694770 6370 ts_tablet_manager.cc:1905] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.694800 6371 raft_consensus.cc:2229] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:50.694836 6369 raft_consensus.cc:2229] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:50.694881 6369 raft_consensus.cc:2258] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.694949 6371 raft_consensus.cc:2258] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.695159 6371 ts_tablet_manager.cc:1905] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.695231 6369 ts_tablet_manager.cc:1905] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.696828 6370 ts_tablet_manager.cc:1918] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:50:50.696834 6371 ts_tablet_manager.cc:1918] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:50:50.696894 6371 log.cc:1196] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/82a13c9aaf9048aa9692ca5bf23ad2a4
I20260806 06:50:50.696990 6370 log.cc:1196] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/82a13c9aaf9048aa9692ca5bf23ad2a4
I20260806 06:50:50.697104 6369 ts_tablet_manager.cc:1918] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:50:50.697165 6369 log.cc:1196] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/82a13c9aaf9048aa9692ca5bf23ad2a4
I20260806 06:50:50.697181 6371 ts_tablet_manager.cc:1939] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a22d73af00954bd6b4160d81833cd91c: Deleting consensus metadata
I20260806 06:50:50.697256 6370 ts_tablet_manager.cc:1939] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting consensus metadata
I20260806 06:50:50.697431 6369 ts_tablet_manager.cc:1939] T 82a13c9aaf9048aa9692ca5bf23ad2a4 P a48694399d75472fa8386e1bdc03d8f1: Deleting consensus metadata
I20260806 06:50:50.697484 5771 catalog_manager.cc:4841] TS a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167): tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (table test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]) successfully deleted
I20260806 06:50:50.697541 5768 catalog_manager.cc:4841] TS 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181): tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (table test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]) successfully deleted
I20260806 06:50:50.697994 5769 catalog_manager.cc:4841] TS a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387): tablet 82a13c9aaf9048aa9692ca5bf23ad2a4 (table test-table [id=1b57951b3e7846bbbc3a9f072eb6a17a]) successfully deleted
Aug 06 06:50:50 dist-test-slave-0kls krb5kdc[5739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999050, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:50:50 dist-test-slave-0kls krb5kdc[5739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999050, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:50.719239 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:50.712087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.3.247.190:34539)
0806 06:50:50.712239 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:50.712242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:50.712318 (+ 76us) server_negotiation.cc:407] Connection header received
0806 06:50:50.712406 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:50.712407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:50.712420 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:50.712499 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:50.713061 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:50.713589 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:50.714372 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:50.714570 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:50.714624 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:50.716740 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:50.716741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:50.716742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:50.716749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:50.718144 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:50.718471 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:50.718473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:50.718473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:50.718532 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:50.718706 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:50.718706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:50.718707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:50.718848 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:50.718930 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:50.719035 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:50:50.719104 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260806 06:50:50.720851 5782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:50.721047 5782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:50.722563 6061 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79cc4c670642daa9edfb4361842b85 (DEFAULT_TABLE table=test-table [id=323975f309b34e29b9589eb96dbd39e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.722612 6200 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79cc4c670642daa9edfb4361842b85 (DEFAULT_TABLE table=test-table [id=323975f309b34e29b9589eb96dbd39e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.722607 5921 tablet_service.cc:1467] Processing CreateTablet for tablet 7d79cc4c670642daa9edfb4361842b85 (DEFAULT_TABLE table=test-table [id=323975f309b34e29b9589eb96dbd39e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.722723 6200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79cc4c670642daa9edfb4361842b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.722723 6061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79cc4c670642daa9edfb4361842b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.722745 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d79cc4c670642daa9edfb4361842b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.723958 6380 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Bootstrap starting.
I20260806 06:50:50.724304 6382 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Bootstrap starting.
I20260806 06:50:50.724363 6380 tablet_bootstrap.cc:654] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.724639 6381 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Bootstrap starting.
I20260806 06:50:50.724730 6382 tablet_bootstrap.cc:654] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.725085 6381 tablet_bootstrap.cc:654] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.725123 6380 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: No bootstrap required, opened a new log
I20260806 06:50:50.725172 6380 ts_tablet_manager.cc:1397] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:50.725349 6380 raft_consensus.cc:348] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.725435 6382 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: No bootstrap required, opened a new log
I20260806 06:50:50.725449 6380 raft_consensus.cc:374] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.725476 6380 raft_consensus.cc:729] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.725479 6382 ts_tablet_manager.cc:1397] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:50.725548 6380 consensus_queue.cc:260] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.725575 6382 raft_consensus.cc:348] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.725634 6382 raft_consensus.cc:374] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.725667 6382 raft_consensus.cc:729] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.725667 6380 ts_tablet_manager.cc:1428] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:50.725739 6382 consensus_queue.cc:260] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.725875 6382 ts_tablet_manager.cc:1428] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:50.726202 6381 tablet_bootstrap.cc:492] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: No bootstrap required, opened a new log
I20260806 06:50:50.726261 6381 ts_tablet_manager.cc:1397] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:50.726397 6381 raft_consensus.cc:348] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.726544 6381 raft_consensus.cc:374] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.726594 6381 raft_consensus.cc:729] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afbef60158d4eb2b648c3c38a85efd0, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.726649 6381 consensus_queue.cc:260] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.726817 6381 ts_tablet_manager.cc:1428] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:50.799257 6350 raft_consensus.cc:482] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:50.799373 6350 raft_consensus.cc:504] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.799608 6350 leader_election.cc:288] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:50.799791 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79cc4c670642daa9edfb4361842b85" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c" is_pre_election: true
I20260806 06:50:50.799947 5941 raft_consensus.cc:2454] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:50.799875 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79cc4c670642daa9edfb4361842b85" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0" is_pre_election: true
I20260806 06:50:50.799995 6081 raft_consensus.cc:2454] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:50.800146 6155 leader_election.cc:302] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:50.800263 6350 raft_consensus.cc:2781] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:50.800344 6350 raft_consensus.cc:482] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:50.800377 6350 raft_consensus.cc:3037] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.801178 6350 raft_consensus.cc:504] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.801321 6350 leader_election.cc:288] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Requested vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:50.801498 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79cc4c670642daa9edfb4361842b85" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c"
I20260806 06:50:50.801501 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d79cc4c670642daa9edfb4361842b85" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0"
I20260806 06:50:50.801576 5941 raft_consensus.cc:3037] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.801575 6081 raft_consensus.cc:3037] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.802331 5941 raft_consensus.cc:2454] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:50.802330 6081 raft_consensus.cc:2454] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:50.802518 6155 leader_election.cc:302] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:50.802616 6350 raft_consensus.cc:2781] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:50.802687 6350 raft_consensus.cc:686] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Becoming Leader. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Running, Role: LEADER
I20260806 06:50:50.802739 6350 consensus_queue.cc:237] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.803397 5782 catalog_manager.cc:5495] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } health_report { overall_health: HEALTHY } } }
I20260806 06:50:50.821606 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:50.819449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.3.247.131:38387)
0806 06:50:50.819578 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:50.819580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:50.819585 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:50:50.819602 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:50.819602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:50.819612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:50.819661 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:50.819809 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:50.820280 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:50.821009 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:50.821195 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:50.821250 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:50.821381 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:50.821405 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:50.821444 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:50.821493 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:50:50.821509 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 06:50:50.823740 6081 raft_consensus.cc:1261] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:50.823758 5941 raft_consensus.cc:1261] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:50.823993 6350 consensus_queue.cc:1035] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:50.824100 6329 consensus_queue.cc:1035] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:50.833410 5783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:50.833513 5783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:50.834157 5783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Sending DeleteTablet for 3 replicas of tablet 7d79cc4c670642daa9edfb4361842b85
I20260806 06:50:50.834404 6061 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d79cc4c670642daa9edfb4361842b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55412
I20260806 06:50:50.834416 6201 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d79cc4c670642daa9edfb4361842b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:52472
I20260806 06:50:50.834411 5921 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d79cc4c670642daa9edfb4361842b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:50 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:45748
I20260806 06:50:50.834494 6370 tablet_replica.cc:330] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: stopping tablet replica
I20260806 06:50:50.834507 6369 tablet_replica.cc:330] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: stopping tablet replica
I20260806 06:50:50.834539 6370 raft_consensus.cc:2229] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:50.834558 6369 raft_consensus.cc:2229] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:50.834578 6370 raft_consensus.cc:2258] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.834613 6369 raft_consensus.cc:2258] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.834595 5783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:50:50.834682 6371 tablet_replica.cc:330] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: stopping tablet replica
I20260806 06:50:50.834730 6371 raft_consensus.cc:2229] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 06:50:50.834731 5783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:50.834764 6371 raft_consensus.cc:2258] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:50.834928 6370 ts_tablet_manager.cc:1905] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.835078 6369 ts_tablet_manager.cc:1905] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.835891 5921 tablet_service.cc:1467] Processing CreateTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f (DEFAULT_TABLE table=test-table [id=126549c3207a406cab7ea375f4f28c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.835904 6201 tablet_service.cc:1467] Processing CreateTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f (DEFAULT_TABLE table=test-table [id=126549c3207a406cab7ea375f4f28c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.836000 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87571c8a452d48cb851c1e2b5f2e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.836018 6201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87571c8a452d48cb851c1e2b5f2e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.836683 6061 tablet_service.cc:1467] Processing CreateTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f (DEFAULT_TABLE table=test-table [id=126549c3207a406cab7ea375f4f28c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:50.836720 6369 ts_tablet_manager.cc:1918] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:50.836771 6369 log.cc:1196] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/7d79cc4c670642daa9edfb4361842b85
I20260806 06:50:50.836838 6370 ts_tablet_manager.cc:1918] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:50.836879 6370 log.cc:1196] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/7d79cc4c670642daa9edfb4361842b85
I20260806 06:50:50.837004 6369 ts_tablet_manager.cc:1939] T 7d79cc4c670642daa9edfb4361842b85 P a48694399d75472fa8386e1bdc03d8f1: Deleting consensus metadata
I20260806 06:50:50.837117 6370 ts_tablet_manager.cc:1939] T 7d79cc4c670642daa9edfb4361842b85 P 3afbef60158d4eb2b648c3c38a85efd0: Deleting consensus metadata
I20260806 06:50:50.837240 6061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87571c8a452d48cb851c1e2b5f2e861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:50.837284 5769 catalog_manager.cc:4841] TS a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387): tablet 7d79cc4c670642daa9edfb4361842b85 (table test-table [id=323975f309b34e29b9589eb96dbd39e6]) successfully deleted
I20260806 06:50:50.837412 6382 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Bootstrap starting.
I20260806 06:50:50.837764 6382 tablet_bootstrap.cc:654] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.837743 6380 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c: Bootstrap starting.
I20260806 06:50:50.838196 6380 tablet_bootstrap.cc:654] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.838308 6382 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: No bootstrap required, opened a new log
I20260806 06:50:50.838366 6382 ts_tablet_manager.cc:1397] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:50.838465 6381 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: Bootstrap starting.
I20260806 06:50:50.838469 6382 raft_consensus.cc:348] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.838562 6382 raft_consensus.cc:374] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.838588 6382 raft_consensus.cc:729] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.838634 6382 consensus_queue.cc:260] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.838727 6382 ts_tablet_manager.cc:1428] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:50.838764 5768 catalog_manager.cc:4841] TS 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181): tablet 7d79cc4c670642daa9edfb4361842b85 (table test-table [id=323975f309b34e29b9589eb96dbd39e6]) successfully deleted
I20260806 06:50:50.838912 6381 tablet_bootstrap.cc:654] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:50.839025 6371 ts_tablet_manager.cc:1905] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:50.839169 6380 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c: No bootstrap required, opened a new log
I20260806 06:50:50.839236 6380 ts_tablet_manager.cc:1397] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:50.839362 6380 raft_consensus.cc:348] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.839509 6380 raft_consensus.cc:374] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.839651 6380 raft_consensus.cc:729] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22d73af00954bd6b4160d81833cd91c, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.839766 6380 consensus_queue.cc:260] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.839974 6380 ts_tablet_manager.cc:1428] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:50:50.840471 6371 ts_tablet_manager.cc:1918] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:50:50.840549 6371 log.cc:1196] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/7d79cc4c670642daa9edfb4361842b85
I20260806 06:50:50.840705 6381 tablet_bootstrap.cc:492] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: No bootstrap required, opened a new log
I20260806 06:50:50.840763 6371 ts_tablet_manager.cc:1939] T 7d79cc4c670642daa9edfb4361842b85 P a22d73af00954bd6b4160d81833cd91c: Deleting consensus metadata
I20260806 06:50:50.840758 6381 ts_tablet_manager.cc:1397] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:50.840903 6381 raft_consensus.cc:348] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.840975 6381 raft_consensus.cc:374] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:50.841035 6381 raft_consensus.cc:729] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afbef60158d4eb2b648c3c38a85efd0, State: Initialized, Role: FOLLOWER
I20260806 06:50:50.841053 5771 catalog_manager.cc:4841] TS a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167): tablet 7d79cc4c670642daa9edfb4361842b85 (table test-table [id=323975f309b34e29b9589eb96dbd39e6]) successfully deleted
I20260806 06:50:50.841089 6381 consensus_queue.cc:260] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.841188 6381 ts_tablet_manager.cc:1428] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:50.928117 6355 raft_consensus.cc:482] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:50.928335 6355 raft_consensus.cc:504] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.928573 6355 leader_election.cc:288] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:50.928776 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87571c8a452d48cb851c1e2b5f2e861f" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c" is_pre_election: true
I20260806 06:50:50.928799 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87571c8a452d48cb851c1e2b5f2e861f" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0" is_pre_election: true
I20260806 06:50:50.928884 5941 raft_consensus.cc:2454] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:50.928954 6081 raft_consensus.cc:2454] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 0.
I20260806 06:50:50.929072 6155 leader_election.cc:302] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a22d73af00954bd6b4160d81833cd91c, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:50.929203 6355 raft_consensus.cc:2781] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:50.929277 6355 raft_consensus.cc:482] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:50.929298 6355 raft_consensus.cc:3037] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.930058 6355 raft_consensus.cc:504] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.930193 6355 leader_election.cc:288] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Requested vote from peers a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167), 3afbef60158d4eb2b648c3c38a85efd0 (127.3.247.130:40181)
I20260806 06:50:50.930377 5941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87571c8a452d48cb851c1e2b5f2e861f" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22d73af00954bd6b4160d81833cd91c"
I20260806 06:50:50.930374 6081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87571c8a452d48cb851c1e2b5f2e861f" candidate_uuid: "a48694399d75472fa8386e1bdc03d8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afbef60158d4eb2b648c3c38a85efd0"
I20260806 06:50:50.930450 5941 raft_consensus.cc:3037] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.930473 6081 raft_consensus.cc:3037] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:50.931059 6081 raft_consensus.cc:2454] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:50.931077 5941 raft_consensus.cc:2454] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a48694399d75472fa8386e1bdc03d8f1 in term 1.
I20260806 06:50:50.931237 6152 leader_election.cc:302] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3afbef60158d4eb2b648c3c38a85efd0, a48694399d75472fa8386e1bdc03d8f1; no voters:
I20260806 06:50:50.931387 6355 raft_consensus.cc:2781] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:50.931468 6355 raft_consensus.cc:686] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Becoming Leader. State: Replica: a48694399d75472fa8386e1bdc03d8f1, State: Running, Role: LEADER
I20260806 06:50:50.931546 6355 consensus_queue.cc:237] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } }
I20260806 06:50:50.932113 5783 catalog_manager.cc:5495] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "a48694399d75472fa8386e1bdc03d8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a48694399d75472fa8386e1bdc03d8f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 38387 } health_report { overall_health: HEALTHY } } }
I20260806 06:50:50.957712 6081 raft_consensus.cc:1261] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:50.957798 5941 raft_consensus.cc:1261] T 87571c8a452d48cb851c1e2b5f2e861f P a22d73af00954bd6b4160d81833cd91c [term 1 FOLLOWER]: Refusing update from remote peer a48694399d75472fa8386e1bdc03d8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:50.957980 6350 consensus_queue.cc:1035] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22d73af00954bd6b4160d81833cd91c" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:50.958114 6329 consensus_queue.cc:1035] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afbef60158d4eb2b648c3c38a85efd0" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:50:50.962028 6348 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:50:52.478539 5784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:52.478654 5784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:52.479578 5784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 588a1bca6e3e4e2ea629d1a3ef892d11: Sending DeleteTablet for 3 replicas of tablet 87571c8a452d48cb851c1e2b5f2e861f
I20260806 06:50:52.479873 6061 tablet_service.cc:1514] Processing DeleteTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:52 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55412
I20260806 06:50:52.479892 5921 tablet_service.cc:1514] Processing DeleteTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:52 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:45748
I20260806 06:50:52.479892 6201 tablet_service.cc:1514] Processing DeleteTablet for tablet 87571c8a452d48cb851c1e2b5f2e861f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:52 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:52472
I20260806 06:50:52.480130 6421 tablet_replica.cc:330] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: stopping tablet replica
I20260806 06:50:52.480195 6421 raft_consensus.cc:2229] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:52.480273 6421 raft_consensus.cc:2258] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:52.480283 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5856
I20260806 06:50:52.481096 6421 ts_tablet_manager.cc:1905] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:52.482470 6419 tablet_replica.cc:330] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: stopping tablet replica
I20260806 06:50:52.482677 6421 ts_tablet_manager.cc:1918] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:50:52.482754 6421 log.cc:1196] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/87571c8a452d48cb851c1e2b5f2e861f
I20260806 06:50:52.482960 6421 ts_tablet_manager.cc:1939] T 87571c8a452d48cb851c1e2b5f2e861f P a48694399d75472fa8386e1bdc03d8f1: Deleting consensus metadata
I20260806 06:50:52.483274 5769 catalog_manager.cc:4841] TS a48694399d75472fa8386e1bdc03d8f1 (127.3.247.131:38387): tablet 87571c8a452d48cb851c1e2b5f2e861f (table test-table [id=126549c3207a406cab7ea375f4f28c5b]) successfully deleted
I20260806 06:50:52.487191 6419 raft_consensus.cc:2229] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:52.487282 6419 raft_consensus.cc:2258] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:50:52.487421 5771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:42167 (error 108)
I20260806 06:50:52.487608 6419 ts_tablet_manager.cc:1905] T 87571c8a452d48cb851c1e2b5f2e861f P 3afbef60158d4eb2b648c3c38a85efd0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:50:52.488178 6423 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:52.487862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:42167 (local address 127.0.0.1:49526)
0806 06:50:52.487982 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:52.487994 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:52.488053 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:52.488108 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:42167: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":75,"threads_started":1}
W20260806 06:50:52.488287 5771 catalog_manager.cc:4584] TS a22d73af00954bd6b4160d81833cd91c (127.3.247.129:42167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 87571c8a452d48cb851c1e2b5f2e861f: Network error: Client connection negotiation failed: client connection to 127.3.247.129:42167: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 06:50:52.488517 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5996
I20260806 06:50:52.493867 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:52.493700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40181 (local address 127.0.0.1:51308)
0806 06:50:52.493761 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:52.493829 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.130:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:50:52.493917 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6136
I20260806 06:50:52.499576 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 5752
2026-08-06T06:50:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5663 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:52 dist-test-slave-0kls krb5kdc[6426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:52 dist-test-slave-0kls krb5kdc[6426](info): set up 2 sockets
Aug 06 06:50:52 dist-test-slave-0kls krb5kdc[6426](info): commencing operation
krb5kdc: starting...
W20260806 06:50:53.593704 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:53 dist-test-slave-0kls krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:53.707886 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40921
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:53.788659 6439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:53.788882 6439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:53.788905 6439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:53.790038 6439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:53.790078 6439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:53.790092 6439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:53.790105 6439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:53.790117 6439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:53.791517 6439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:53.791759 6439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:53.791989 6439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:53.797181 6447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:53.797248 6444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:53.797308 6445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:53.797554 6439 server_base.cc:1034] running on GCE node
I20260806 06:50:53.797839 6439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:53.798151 6439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:53.799294 6439 hybrid_clock.cc:648] HybridClock initialized: now 1785999053799274 us; error 37 us; skew 500 ppm
Aug 06 06:50:53 dist-test-slave-0kls krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999053, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:53.801676 6439 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:53.802198 6439 webserver.cc:460] Webserver started at http://127.3.247.190:34395/ using document root <none> and password file <none>
I20260806 06:50:53.802382 6439 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:53.802423 6439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:53.802510 6439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:53.803527 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "9c6c4e95e0d44f64bed6597696399d21"
format_stamp: "Formatted at 2026-08-06 06:50:53 on dist-test-slave-0kls"
server_key: "fd2b8c31c8a6b12c4e21a8cabd6292e5"
server_key_iv: "5077aeb3da713948280b0f8f3a6e0adc"
server_key_version: "encryptionkey@0"
I20260806 06:50:53.803829 6439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "9c6c4e95e0d44f64bed6597696399d21"
format_stamp: "Formatted at 2026-08-06 06:50:53 on dist-test-slave-0kls"
server_key: "fd2b8c31c8a6b12c4e21a8cabd6292e5"
server_key_iv: "5077aeb3da713948280b0f8f3a6e0adc"
server_key_version: "encryptionkey@0"
I20260806 06:50:53.805212 6439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:53.806006 6454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:53.806182 6439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:53.806262 6439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "9c6c4e95e0d44f64bed6597696399d21"
format_stamp: "Formatted at 2026-08-06 06:50:53 on dist-test-slave-0kls"
server_key: "fd2b8c31c8a6b12c4e21a8cabd6292e5"
server_key_iv: "5077aeb3da713948280b0f8f3a6e0adc"
server_key_version: "encryptionkey@0"
I20260806 06:50:53.806320 6439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:53.822834 6439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:53.825141 6439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:53.825251 6439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:53.828959 6439 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40921
I20260806 06:50:53.829162 6506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40921 every 8 connection(s)
I20260806 06:50:53.829347 6439 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:53.830319 6507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:53.832468 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6439
I20260806 06:50:53.832573 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:53.832741 4062 external_mini_cluster.cc:1428] Setting key d701a61be28c9b06640b82e09748b8cf
I20260806 06:50:53.832726 6507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: Bootstrap starting.
I20260806 06:50:53.833432 6507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:53.833740 6507 log.cc:824] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:53.834438 6507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: No bootstrap required, opened a new log
I20260806 06:50:53.835768 6507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } }
I20260806 06:50:53.835924 6507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:53.835980 6507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6c4e95e0d44f64bed6597696399d21, State: Initialized, Role: FOLLOWER
I20260806 06:50:53.836104 6507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } }
I20260806 06:50:53.836201 6507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:53.836244 6507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:53.836316 6507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:53.836871 6507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } }
I20260806 06:50:53.837018 6507 leader_election.cc:302] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c6c4e95e0d44f64bed6597696399d21; no voters:
I20260806 06:50:53.837224 6507 leader_election.cc:288] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:53.837249 6512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:53.837364 6512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [term 1 LEADER]: Becoming Leader. State: Replica: 9c6c4e95e0d44f64bed6597696399d21, State: Running, Role: LEADER
I20260806 06:50:53.837484 6507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:53.837523 6512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } }
I20260806 06:50:53.837868 6512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c6c4e95e0d44f64bed6597696399d21. Latest consensus state: current_term: 1 leader_uuid: "9c6c4e95e0d44f64bed6597696399d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } } }
I20260806 06:50:53.837884 6513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c6c4e95e0d44f64bed6597696399d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6c4e95e0d44f64bed6597696399d21" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40921 } } }
I20260806 06:50:53.838002 6512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:53.838799 6513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:53.839280 6528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:53.839349 6528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:53.839427 6520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:53.839617 6520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:50:53 dist-test-slave-0kls krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:53.841065 6520 catalog_manager.cc:1347] Generated new cluster ID: 601c3a1a18544dbea27ff4efeb114ad5
I20260806 06:50:53.841114 6520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:53.844250 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:53.833262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.3.247.190:40921)
0806 06:50:53.833435 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:53.833443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:53.833468 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:50:53.834009 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:53.834016 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:53.834065 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:53.834223 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:53.834913 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:53.836338 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:53.837216 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:53.837851 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:53.838275 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:53.840910 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:53.840914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:53.840915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:53.840925 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:53.842704 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:53.843097 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:53.843100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:53.843100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:53.843189 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:53.843404 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:53.843405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:53.843405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:53.843639 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:53.843806 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:53.843910 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:50:53.844051 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 06:50:53.853662 6520 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:53.854250 6520 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:53.860594 6520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c6c4e95e0d44f64bed6597696399d21: Generated new TSK 0
I20260806 06:50:53.860805 6520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:53.892318 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:53.999172 6533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:53.999414 6533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:53.999462 6533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:54.000648 6533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:54.000717 6533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:54.000765 6533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:54.002460 6533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:54.002750 6533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:54.003023 6533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:54.009841 6541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.009841 6538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.010170 6539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:54.011111 6533 server_base.cc:1034] running on GCE node
I20260806 06:50:54.011328 6533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:54.011593 6533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:54.012737 6533 hybrid_clock.cc:648] HybridClock initialized: now 1785999054012725 us; error 32 us; skew 500 ppm
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:54.015570 6533 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:54.016106 6533 webserver.cc:460] Webserver started at http://127.3.247.129:42451/ using document root <none> and password file <none>
I20260806 06:50:54.016422 6533 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:54.016487 6533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:54.016644 6533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:54.018250 6533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "696771235179426a8fec9881c2314694"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "f5a018bf5ab2df47bf1c11b8b18206a9"
server_key_iv: "ed05946e993aa00471facf489beea432"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.018702 6533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "696771235179426a8fec9881c2314694"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "f5a018bf5ab2df47bf1c11b8b18206a9"
server_key_iv: "ed05946e993aa00471facf489beea432"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.020545 6533 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:54.021557 6548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.021765 6533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:54.021934 6533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "696771235179426a8fec9881c2314694"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "f5a018bf5ab2df47bf1c11b8b18206a9"
server_key_iv: "ed05946e993aa00471facf489beea432"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.022039 6533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:54.043622 6533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:54.045687 6533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:54.045938 6533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:54.046211 6533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:54.046568 6533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:54.046629 6533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.046674 6533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:54.046720 6533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.052976 6533 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:45449
I20260806 06:50:54.053062 6661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:45449 every 8 connection(s)
I20260806 06:50:54.053413 6533 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:54.058456 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6533
I20260806 06:50:54.058564 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:54.058756 4062 external_mini_cluster.cc:1428] Setting key df8a32957098f56d95363b929ba82c83
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:54.063891 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.054232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:57779 (local address 127.3.247.190:40921)
0806 06:50:54.054303 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:54.054305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:54.054503 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:50:54.054977 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:54.054978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:54.054987 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:54.055053 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:54.056011 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.056550 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.057502 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.057721 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.057801 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.060724 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:54.060725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:54.060725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:54.060732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:54.062441 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.062855 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.062856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.062857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.062918 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.063191 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.063192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.063193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.063340 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:54.063414 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:54.063692 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 06:50:54.063790 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:50:54.063941 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.054199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40921 (local address 127.3.247.129:57779)
0806 06:50:54.054373 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:54.054416 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:54.054898 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:54.055104 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:54.055107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:54.055700 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:54.055885 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.055890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.056629 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.056629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.057419 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.057421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.057851 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.057852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.057990 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.058652 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:54.058661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:54.060618 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:54.062538 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.062545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.062547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.062781 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.062984 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.062984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.062985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.063112 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.063417 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:54.063424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:54.063553 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:50:54.063642 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:50:54.063764 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260806 06:50:54.064525 6662 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40921
I20260806 06:50:54.064628 6662 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:54.064823 6662 heartbeater.cc:511] Master 127.3.247.190:40921 requested a full tablet report, sending...
I20260806 06:50:54.065433 6471 ts_manager.cc:194] Registered new tserver with Master: 696771235179426a8fec9881c2314694 (127.3.247.129:45449)
I20260806 06:50:54.066143 6471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:57779
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:54.083974 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:54.164331 6667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:54.164554 6667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:54.164589 6667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:54.165735 6667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:54.165820 6667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:54.165879 6667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:54.167276 6667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:54.167529 6667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:54.167753 6667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:54.172623 6672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.172703 6675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.172634 6673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:54.172888 6667 server_base.cc:1034] running on GCE node
I20260806 06:50:54.173103 6667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:54.173367 6667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:54.174507 6667 hybrid_clock.cc:648] HybridClock initialized: now 1785999054174491 us; error 37 us; skew 500 ppm
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:54.176987 6667 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:54.177394 6667 webserver.cc:460] Webserver started at http://127.3.247.130:40529/ using document root <none> and password file <none>
I20260806 06:50:54.177635 6667 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:54.177680 6667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:54.177860 6667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:54.178933 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "b7b7aa25a8ca4d83a803729036601ffc"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "ad32acf3bab94f8ec351770f711b99da"
server_key_iv: "ab64f6bde4e647a36e1476d7d3c6e2f1"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.179280 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "b7b7aa25a8ca4d83a803729036601ffc"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "ad32acf3bab94f8ec351770f711b99da"
server_key_iv: "ab64f6bde4e647a36e1476d7d3c6e2f1"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.180608 6667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:54.181278 6682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.181458 6667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:54.181547 6667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "b7b7aa25a8ca4d83a803729036601ffc"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "ad32acf3bab94f8ec351770f711b99da"
server_key_iv: "ab64f6bde4e647a36e1476d7d3c6e2f1"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.181632 6667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:54.198907 6667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:54.201252 6667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:54.201429 6667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:54.201676 6667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:54.202044 6667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:54.202097 6667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.202140 6667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:54.202168 6667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.208297 6667 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:43207
I20260806 06:50:54.208343 6795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:43207 every 8 connection(s)
I20260806 06:50:54.208706 6667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:54.217918 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.209453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:39283 (local address 127.3.247.190:40921)
0806 06:50:54.209497 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:54.209499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:54.209694 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:50:54.210302 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:54.210304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:54.210316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:54.210373 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:54.211334 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.211847 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.212893 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.213111 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.213198 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.215313 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:54.215315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:54.215316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:54.215322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:54.216743 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.217102 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.217103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.217103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.217153 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.217357 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.217357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.217358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.217455 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:54.217505 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:54.217734 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:50:54.217852 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:54.218120 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.209424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40921 (local address 127.3.247.130:39283)
0806 06:50:54.209561 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:54.209607 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:54.210194 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:54.210445 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:54.210449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:54.211028 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:54.211222 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.211227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.211984 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.211985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.212800 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.212802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.213208 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.213209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.213338 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.213887 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:54.213895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:54.215224 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:54.216815 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.216820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.216822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.217038 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.217210 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.217211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.217211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.217303 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.217511 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:54.217515 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:54.217625 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 06:50:54.217707 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:50:54.217876 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260806 06:50:54.218648 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6667
I20260806 06:50:54.218741 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:54.218937 4062 external_mini_cluster.cc:1428] Setting key 871886d9909365a4e97b5d255b31b3f0
I20260806 06:50:54.219245 6796 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40921
I20260806 06:50:54.219369 6796 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:54.219547 6796 heartbeater.cc:511] Master 127.3.247.190:40921 requested a full tablet report, sending...
I20260806 06:50:54.219944 6471 ts_manager.cc:194] Registered new tserver with Master: b7b7aa25a8ca4d83a803729036601ffc (127.3.247.130:43207)
I20260806 06:50:54.220463 6471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:39283
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:54.242815 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:54.325886 6801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:54.326155 6801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:54.326196 6801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:54.327430 6801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:54.327499 6801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:54.327546 6801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:54.329064 6801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:54.329360 6801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:54.329614 6801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:54.334789 6801 server_base.cc:1034] running on GCE node
W20260806 06:50:54.334712 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.334738 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:54.334743 6809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:54.335160 6801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:54.335453 6801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:54.336603 6801 hybrid_clock.cc:648] HybridClock initialized: now 1785999054336595 us; error 32 us; skew 500 ppm
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:54.339315 6801 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:54.339789 6801 webserver.cc:460] Webserver started at http://127.3.247.131:40431/ using document root <none> and password file <none>
I20260806 06:50:54.340056 6801 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:54.340102 6801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:54.340262 6801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:54.341305 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "c4425bbe673948768750d408ecfb3f92"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "aa0d5a5aa3550a20beef9041dcf04820"
server_key_iv: "a4b1b9916519ee652905dc2c110b2256"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.341663 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "c4425bbe673948768750d408ecfb3f92"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "aa0d5a5aa3550a20beef9041dcf04820"
server_key_iv: "a4b1b9916519ee652905dc2c110b2256"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.342990 6801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:54.343709 6816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.343883 6801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:54.343987 6801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "c4425bbe673948768750d408ecfb3f92"
format_stamp: "Formatted at 2026-08-06 06:50:54 on dist-test-slave-0kls"
server_key: "aa0d5a5aa3550a20beef9041dcf04820"
server_key_iv: "a4b1b9916519ee652905dc2c110b2256"
server_key_version: "encryptionkey@0"
I20260806 06:50:54.344060 6801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:54.352265 6801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:54.354501 6801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:54.354655 6801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:54.354866 6801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:54.355170 6801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:54.355224 6801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.355265 6801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:54.355293 6801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:54.361416 6801 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:40163
I20260806 06:50:54.361464 6929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:40163 every 8 connection(s)
I20260806 06:50:54.361825 6801 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:50:54.367311 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6801
I20260806 06:50:54.367384 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:54.367579 4062 external_mini_cluster.cc:1428] Setting key 80277070897f200a94c5ba6bf6da620a
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999054, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:54.371419 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.362558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:44237 (local address 127.3.247.190:40921)
0806 06:50:54.362618 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:54.362620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:54.362833 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:50:54.363373 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:54.363374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:54.363387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:54.363437 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:54.364427 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.364930 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.366018 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.366217 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.366319 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.368778 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:54.368778 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:54.368779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:54.368785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:54.370271 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.370629 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.370629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.370630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.370681 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:54.370892 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:54.370893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:54.370893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:54.371014 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:54.371065 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:54.371283 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 06:50:54.371346 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:50:54.371671 6932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:54.362539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40921 (local address 127.3.247.131:44237)
0806 06:50:54.362710 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:54.362747 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:54.363258 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:54.363524 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:54.363528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:54.364119 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:54.364320 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.364325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.365029 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.365030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.365920 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:54.365923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:54.366326 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:54.366327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:54.366478 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:54.366965 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:54.366974 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:54.368701 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:54.370341 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.370346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.370348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.370563 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.370738 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:54.370739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:54.370739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:54.370840 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:54.371071 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:54.371075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:54.371190 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 06:50:54.371273 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:50:54.371392 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260806 06:50:54.372161 6930 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40921
I20260806 06:50:54.372260 6930 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:54.372437 6930 heartbeater.cc:511] Master 127.3.247.190:40921 requested a full tablet report, sending...
I20260806 06:50:54.372828 6471 ts_manager.cc:194] Registered new tserver with Master: c4425bbe673948768750d408ecfb3f92 (127.3.247.131:40163)
I20260806 06:50:54.373214 6471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:44237
I20260806 06:50:54.379464 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:50:54.391067 6510 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:54.390122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.3.247.190:40921)
0806 06:50:54.390171 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:54.390173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:54.390289 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:50:54.390368 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:54.390369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:54.390378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:54.390422 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:54.390957 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56846: BlockingRecv error: recv got EOF from 127.0.0.1:56846 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:50:54.391919 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6533
I20260806 06:50:54.396750 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6667
I20260806 06:50:54.400835 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6801
I20260806 06:50:54.404887 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6439
2026-08-06T06:50:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1903 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6942](info): set up 2 sockets
Aug 06 06:50:54 dist-test-slave-0kls krb5kdc[6942](info): commencing operation
krb5kdc: starting...
W20260806 06:50:55.437690 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[6942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:55.552949 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:32775
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:55.635962 6955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:55.636219 6955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:55.636257 6955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:55.637435 6955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:55.637508 6955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:55.637532 6955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:55.637562 6955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:55.637593 6955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:55.639130 6955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:32775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:55.639417 6955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:55.639657 6955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:55.644765 6963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:55.644979 6960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:55.644865 6961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:55.644882 6955 server_base.cc:1034] running on GCE node
I20260806 06:50:55.645359 6955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:55.645613 6955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:55.646754 6955 hybrid_clock.cc:648] HybridClock initialized: now 1785999055646744 us; error 36 us; skew 500 ppm
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[6942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999055, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:55.649309 6955 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:55.649748 6955 webserver.cc:460] Webserver started at http://127.3.247.190:36465/ using document root <none> and password file <none>
I20260806 06:50:55.650040 6955 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:55.650116 6955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:55.650267 6955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:55.651381 6955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "18bd7d4cdb4545cf8339ed662319c6f3"
format_stamp: "Formatted at 2026-08-06 06:50:55 on dist-test-slave-0kls"
server_key: "5e5f7fbd888b267f910810b4232615de"
server_key_iv: "cdd39e136ba4161920695c3f0fed876a"
server_key_version: "encryptionkey@0"
I20260806 06:50:55.651758 6955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "18bd7d4cdb4545cf8339ed662319c6f3"
format_stamp: "Formatted at 2026-08-06 06:50:55 on dist-test-slave-0kls"
server_key: "5e5f7fbd888b267f910810b4232615de"
server_key_iv: "cdd39e136ba4161920695c3f0fed876a"
server_key_version: "encryptionkey@0"
I20260806 06:50:55.653157 6955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:55.653923 6970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:55.654091 6955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:55.654196 6955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "18bd7d4cdb4545cf8339ed662319c6f3"
format_stamp: "Formatted at 2026-08-06 06:50:55 on dist-test-slave-0kls"
server_key: "5e5f7fbd888b267f910810b4232615de"
server_key_iv: "cdd39e136ba4161920695c3f0fed876a"
server_key_version: "encryptionkey@0"
I20260806 06:50:55.654287 6955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:55.672601 6955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:55.674604 6955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:55.674777 6955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:55.678854 6955 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:32775
I20260806 06:50:55.678892 7022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:32775 every 8 connection(s)
I20260806 06:50:55.679257 6955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:55.680169 7023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:55.682508 7023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: Bootstrap starting.
I20260806 06:50:55.683121 7023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:55.683429 7023 log.cc:824] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:55.684125 7023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: No bootstrap required, opened a new log
I20260806 06:50:55.685700 7023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } }
I20260806 06:50:55.685868 7023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:55.685894 7023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bd7d4cdb4545cf8339ed662319c6f3, State: Initialized, Role: FOLLOWER
I20260806 06:50:55.686090 7023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } }
I20260806 06:50:55.686154 7023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:55.686175 7023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:55.686244 7023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:55.687052 7023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } }
I20260806 06:50:55.687163 7023 leader_election.cc:302] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18bd7d4cdb4545cf8339ed662319c6f3; no voters:
I20260806 06:50:55.687350 7023 leader_election.cc:288] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:55.687397 7026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:55.687605 7026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 18bd7d4cdb4545cf8339ed662319c6f3, State: Running, Role: LEADER
I20260806 06:50:55.687620 7023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:55.687707 7026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } }
I20260806 06:50:55.687964 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 6955
I20260806 06:50:55.688087 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:55.688287 4062 external_mini_cluster.cc:1428] Setting key 74755597a2a10c55bb223a9e090c3ff4
I20260806 06:50:55.688513 7028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18bd7d4cdb4545cf8339ed662319c6f3. Latest consensus state: current_term: 1 leader_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } } }
I20260806 06:50:55.688596 7028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:55.688783 7027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18bd7d4cdb4545cf8339ed662319c6f3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 32775 } } }
I20260806 06:50:55.688869 7027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:55.689716 7043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:55.689776 7043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:55.689908 7036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:55.690124 7036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:55.693734 7036 catalog_manager.cc:1347] Generated new cluster ID: 1b491a6cdba44e329054d4cc33bf22c9
I20260806 06:50:55.693843 7036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[6942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:55.699646 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:55.688920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.3.247.190:32775)
0806 06:50:55.689195 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:55.689206 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:55.689241 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:50:55.689422 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:55.689429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:55.689503 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:55.690534 (+ 1031us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:55.690646 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:55.691681 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:55.692437 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:55.692880 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:55.693009 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:55.695801 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:55.695806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:55.695807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:55.695820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:55.697911 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:55.698324 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:55.698327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:55.698328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:55.698444 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:55.698651 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:55.698651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:55.698652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:55.699027 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:55.699193 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:55.699267 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:50:55.699413 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":65,"threads_started":1}
I20260806 06:50:55.700659 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:55.704128 7036 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:55.704890 7036 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:55.708297 7036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18bd7d4cdb4545cf8339ed662319c6f3: Generated new TSK 0
I20260806 06:50:55.708490 7036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:55.716849 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:55.712925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.3.247.190:32775)
0806 06:50:55.712995 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:55.712997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:55.713198 (+ 201us) server_negotiation.cc:407] Connection header received
0806 06:50:55.713227 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:55.713228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:55.713239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:55.713285 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:55.713745 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:55.714318 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:55.715400 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:55.715614 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:55.715663 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:55.716342 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:55.716343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:55.716343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:55.716570 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:55.716587 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:55.716593 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:55.716639 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:55.716725 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:50:55.716749 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:50:55.719394 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 6955
2026-08-06T06:50:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1312 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[7056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[7056](info): set up 2 sockets
Aug 06 06:50:55 dist-test-slave-0kls krb5kdc[7056](info): commencing operation
krb5kdc: starting...
W20260806 06:50:56.753868 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:56 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:56.873829 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33465
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:56.954512 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:56.954763 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:56.954802 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:56.955976 7069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:56.956048 7069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:56.956070 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:56.956091 7069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:56.956121 7069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:56.957600 7069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:56.957896 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:56.958141 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:56.962934 7074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:56.962947 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:56.963052 7075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:56.963294 7069 server_base.cc:1034] running on GCE node
I20260806 06:50:56.963521 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:56.963807 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:56.964967 7069 hybrid_clock.cc:648] HybridClock initialized: now 1785999056964941 us; error 39 us; skew 500 ppm
Aug 06 06:50:56 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999056, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:56.967689 7069 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:56.968137 7069 webserver.cc:460] Webserver started at http://127.3.247.190:35269/ using document root <none> and password file <none>
I20260806 06:50:56.968354 7069 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:56.968397 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:56.968492 7069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:56.969465 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "bbb8df0e53464966a5c345477db4c479"
format_stamp: "Formatted at 2026-08-06 06:50:56 on dist-test-slave-0kls"
server_key: "951b91b5935fc11b0812d48003596ca8"
server_key_iv: "e1049db550cd05d22035550f2bfd34df"
server_key_version: "encryptionkey@0"
I20260806 06:50:56.969766 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "bbb8df0e53464966a5c345477db4c479"
format_stamp: "Formatted at 2026-08-06 06:50:56 on dist-test-slave-0kls"
server_key: "951b91b5935fc11b0812d48003596ca8"
server_key_iv: "e1049db550cd05d22035550f2bfd34df"
server_key_version: "encryptionkey@0"
I20260806 06:50:56.971083 7069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:56.971777 7084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:56.971961 7069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:56.972061 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "bbb8df0e53464966a5c345477db4c479"
format_stamp: "Formatted at 2026-08-06 06:50:56 on dist-test-slave-0kls"
server_key: "951b91b5935fc11b0812d48003596ca8"
server_key_iv: "e1049db550cd05d22035550f2bfd34df"
server_key_version: "encryptionkey@0"
I20260806 06:50:56.972138 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:57.022068 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:57.024209 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:57.024389 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:57.028375 7069 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:33465
I20260806 06:50:57.028447 7136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:33465 every 8 connection(s)
I20260806 06:50:57.028810 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:57.029275 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7069
I20260806 06:50:57.029394 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:57.029654 4062 external_mini_cluster.cc:1428] Setting key bf31bb9fb975eb312238feaa29734682
I20260806 06:50:57.030120 7137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:57.033393 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479: Bootstrap starting.
I20260806 06:50:57.034080 7137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:57.034397 7137 log.cc:824] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:57.035109 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479: No bootstrap required, opened a new log
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:57.036974 7137 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } }
I20260806 06:50:57.037189 7137 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:57.037233 7137 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb8df0e53464966a5c345477db4c479, State: Initialized, Role: FOLLOWER
I20260806 06:50:57.037400 7137 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } }
I20260806 06:50:57.037506 7137 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:57.037551 7137 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:57.037607 7137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:57.038230 7137 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } }
I20260806 06:50:57.038353 7137 leader_election.cc:302] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb8df0e53464966a5c345477db4c479; no voters:
I20260806 06:50:57.038555 7137 leader_election.cc:288] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:57.038611 7142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:57.038765 7142 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [term 1 LEADER]: Becoming Leader. State: Replica: bbb8df0e53464966a5c345477db4c479, State: Running, Role: LEADER
I20260806 06:50:57.038836 7137 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:57.038861 7142 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } }
I20260806 06:50:57.039206 7142 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb8df0e53464966a5c345477db4c479. Latest consensus state: current_term: 1 leader_uuid: "bbb8df0e53464966a5c345477db4c479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } } }
I20260806 06:50:57.039304 7142 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:57.039215 7144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb8df0e53464966a5c345477db4c479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb8df0e53464966a5c345477db4c479" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33465 } } }
I20260806 06:50:57.039461 7144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:57.040148 7149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:57.040616 7149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:57.041177 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.030224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.3.247.190:33465)
0806 06:50:57.030422 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:57.030430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:57.030456 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:50:57.030898 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:57.030905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:57.030951 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:57.031073 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:57.031864 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.032774 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.033849 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.034318 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.034405 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.036767 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:57.036771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:57.036772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:57.036786 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:57.038848 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.039307 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.039310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.039310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.039420 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.039629 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.039630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.039630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.040000 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:57.040150 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:57.040759 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 06:50:57.040898 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260806 06:50:57.046303 7149 catalog_manager.cc:1347] Generated new cluster ID: e5e84ebfed6f490eb6e5ea8b134a8e04
I20260806 06:50:57.046370 7149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:57.056874 7149 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:57.057489 7149 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:57.066922 7149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb8df0e53464966a5c345477db4c479: Generated new TSK 0
I20260806 06:50:57.067137 7149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:57.104395 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:57.186013 7163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:57.186223 7163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:57.186249 7163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:57.187389 7163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:57.187427 7163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:57.187455 7163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:57.188925 7163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:57.189121 7163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:57.189304 7163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:57.194366 7168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:57.194443 7169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.194456 7163 server_base.cc:1034] running on GCE node
W20260806 06:50:57.194366 7171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.194826 7163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:57.195089 7163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:57.196259 7163 hybrid_clock.cc:648] HybridClock initialized: now 1785999057196254 us; error 34 us; skew 500 ppm
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:57.198918 7163 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:57.199407 7163 webserver.cc:460] Webserver started at http://127.3.247.129:40919/ using document root <none> and password file <none>
I20260806 06:50:57.199594 7163 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:57.199635 7163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:57.199719 7163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:57.200714 7163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "14f8637524a2498f8b02500db749ea79"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "204bd70b98c8236f3d031ba8508ccc7f"
server_key_iv: "f8ba329fc0280bff445b5713eaa425ab"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.201079 7163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "14f8637524a2498f8b02500db749ea79"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "204bd70b98c8236f3d031ba8508ccc7f"
server_key_iv: "f8ba329fc0280bff445b5713eaa425ab"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.202457 7163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:57.203136 7178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.203310 7163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:57.203397 7163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "14f8637524a2498f8b02500db749ea79"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "204bd70b98c8236f3d031ba8508ccc7f"
server_key_iv: "f8ba329fc0280bff445b5713eaa425ab"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.203451 7163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:57.210742 7163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:57.212508 7163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:57.212630 7163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:57.212802 7163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:57.213037 7163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:57.213064 7163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.213085 7163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:57.213097 7163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.218557 7163 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:38651
I20260806 06:50:57.218637 7291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:38651 every 8 connection(s)
I20260806 06:50:57.218950 7163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:57.219256 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7163
I20260806 06:50:57.219350 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:57.219573 4062 external_mini_cluster.cc:1428] Setting key 0a61fd21b2e20945172931827aa6e655
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:50:57.228642 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.219709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:56729 (local address 127.3.247.190:33465)
0806 06:50:57.219869 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:57.219872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:57.220047 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:50:57.220643 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:57.220644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:57.220656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:57.220712 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:57.221931 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.222422 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.223454 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.223669 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.223756 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.225837 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:57.225838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:57.225839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:57.225847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:57.227291 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.227694 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.227696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.227696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.227757 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.228011 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.228012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.228012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.228176 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:57.228246 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:57.228500 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 06:50:57.228574 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 06:50:57.228816 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.219711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33465 (local address 127.3.247.129:56729)
0806 06:50:57.219921 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:57.219970 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:57.220543 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:57.220783 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:57.220787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:57.221503 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:57.221835 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.221842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.222507 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.222508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.223357 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.223359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.223766 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.223767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.223889 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.224397 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:57.224405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:57.225707 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:57.227382 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.227389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.227391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.227617 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.227833 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.227834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.227834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.227943 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.228250 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:57.228257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:57.228380 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:50:57.228476 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:50:57.228603 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
I20260806 06:50:57.229378 7292 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33465
I20260806 06:50:57.229528 7292 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
I20260806 06:50:57.229746 7292 heartbeater.cc:511] Master 127.3.247.190:33465 requested a full tablet report, sending...
I20260806 06:50:57.230322 7101 ts_manager.cc:194] Registered new tserver with Master: 14f8637524a2498f8b02500db749ea79 (127.3.247.129:38651)
I20260806 06:50:57.230989 7101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:56729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:57.244822 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:57.324795 7297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:57.325037 7297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:57.325062 7297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:57.326195 7297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:57.326232 7297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:57.326267 7297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:57.327663 7297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:57.327852 7297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:57.328032 7297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:57.333076 7303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:57.333178 7302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.333315 7297 server_base.cc:1034] running on GCE node
W20260806 06:50:57.333076 7305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.333583 7297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:57.333837 7297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:57.335008 7297 hybrid_clock.cc:648] HybridClock initialized: now 1785999057334996 us; error 31 us; skew 500 ppm
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:57.337399 7297 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:57.337920 7297 webserver.cc:460] Webserver started at http://127.3.247.130:35521/ using document root <none> and password file <none>
I20260806 06:50:57.338158 7297 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:57.338248 7297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:57.338405 7297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:57.339757 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "09c9ea3bf292453daaea264615243e30"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "173a8ba47565671990165ce482b36ab0"
server_key_iv: "977091cd6326e07c8ab4a97f32578085"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.340158 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "09c9ea3bf292453daaea264615243e30"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "173a8ba47565671990165ce482b36ab0"
server_key_iv: "977091cd6326e07c8ab4a97f32578085"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.341560 7297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:57.342406 7312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.342593 7297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:57.342693 7297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "09c9ea3bf292453daaea264615243e30"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "173a8ba47565671990165ce482b36ab0"
server_key_iv: "977091cd6326e07c8ab4a97f32578085"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.342784 7297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:57.352731 7297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:57.354661 7297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:57.354826 7297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:57.355063 7297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:57.355378 7297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:57.355453 7297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.355495 7297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:57.355538 7297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.361733 7297 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:40291
I20260806 06:50:57.361771 7425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:40291 every 8 connection(s)
I20260806 06:50:57.362125 7297 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:57.369725 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7297
I20260806 06:50:57.369872 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:57.370087 4062 external_mini_cluster.cc:1428] Setting key 3d10a18e5f4f4d33ba3c76cea899409a
I20260806 06:50:57.371352 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.362876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:50495 (local address 127.3.247.190:33465)
0806 06:50:57.362928 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:57.362930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:57.363104 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:50:57.363615 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:57.363616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:57.363625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:57.363686 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:57.364610 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.365092 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.366152 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.366336 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.366389 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.368379 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:57.368380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:57.368380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:57.368387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:57.369778 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.370301 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.370302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.370302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.370385 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.370698 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.370699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.370700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.370823 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:57.370903 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:57.371185 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:50:57.371261 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:50:57.371567 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.362829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33465 (local address 127.3.247.130:50495)
0806 06:50:57.362986 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:57.363029 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:57.363517 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:57.363762 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:57.363765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:57.364326 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:57.364529 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.364533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.365170 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.365171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.366079 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.366081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.366406 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.366406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.366513 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.366978 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:57.366986 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:57.368293 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:57.369929 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.369936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.369938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.370220 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.370462 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.370463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.370463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.370613 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.370895 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:57.370901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:57.371047 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 06:50:57.371154 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:50:57.371313 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260806 06:50:57.372151 7426 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33465
I20260806 06:50:57.372257 7426 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:57.372453 7426 heartbeater.cc:511] Master 127.3.247.190:33465 requested a full tablet report, sending...
I20260806 06:50:57.372808 7101 ts_manager.cc:194] Registered new tserver with Master: 09c9ea3bf292453daaea264615243e30 (127.3.247.130:40291)
I20260806 06:50:57.373193 7101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:50495
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:57.396507 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:57.478168 7431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:57.478372 7431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:57.478396 7431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:57.479595 7431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:57.479641 7431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:57.479668 7431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:57.481211 7431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:57.481446 7431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:57.481722 7431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:57.486757 7436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:57.486900 7439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.486927 7431 server_base.cc:1034] running on GCE node
W20260806 06:50:57.487093 7437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:57.487320 7431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:57.487589 7431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:57.488751 7431 hybrid_clock.cc:648] HybridClock initialized: now 1785999057488769 us; error 57 us; skew 500 ppm
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:57.491359 7431 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:57.491786 7431 webserver.cc:460] Webserver started at http://127.3.247.131:40919/ using document root <none> and password file <none>
I20260806 06:50:57.492030 7431 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:57.492075 7431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:57.492214 7431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:57.493276 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "7a6c4371d97a46ffa046ff61fd79aa6b"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "a182e81de94773af7057cdbe7d7d6f7c"
server_key_iv: "98fc1b3fc6fa687f32b5941e85658742"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.493629 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "7a6c4371d97a46ffa046ff61fd79aa6b"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "a182e81de94773af7057cdbe7d7d6f7c"
server_key_iv: "98fc1b3fc6fa687f32b5941e85658742"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.495090 7431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:57.495925 7446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.496136 7431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:50:57.496250 7431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "7a6c4371d97a46ffa046ff61fd79aa6b"
format_stamp: "Formatted at 2026-08-06 06:50:57 on dist-test-slave-0kls"
server_key: "a182e81de94773af7057cdbe7d7d6f7c"
server_key_iv: "98fc1b3fc6fa687f32b5941e85658742"
server_key_version: "encryptionkey@0"
I20260806 06:50:57.496337 7431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:57.506649 7431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:57.509058 7431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:57.509228 7431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:57.509483 7431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:57.509859 7431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:57.509914 7431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.509958 7431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:57.509987 7431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:57.515756 7431 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:32787
I20260806 06:50:57.515794 7559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:32787 every 8 connection(s)
I20260806 06:50:57.516152 7431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:50:57.521641 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7431
I20260806 06:50:57.521761 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:57.521967 4062 external_mini_cluster.cc:1428] Setting key 8ba8c237c36d59855a7de79457574556
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:57.525915 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.516967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:45711 (local address 127.3.247.190:33465)
0806 06:50:57.517040 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:57.517042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:57.517274 (+ 232us) server_negotiation.cc:407] Connection header received
0806 06:50:57.517829 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:57.517831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:57.517845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:57.517916 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:57.518887 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.519369 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.520387 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.520586 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.520646 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.523204 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:57.523206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:57.523206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:57.523215 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:57.524415 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.524863 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.524864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.524865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.524929 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.525206 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.525207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.525207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.525332 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:57.525412 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:57.525703 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 06:50:57.525819 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:50:57.526165 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.516935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33465 (local address 127.3.247.131:45711)
0806 06:50:57.517132 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:57.517176 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:57.517702 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:57.517998 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:57.518001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:57.518585 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:57.518791 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.518796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.519454 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.519455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.520290 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:57.520292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.520668 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.520668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:57.520773 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.521391 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:57.521401 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:57.523104 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:57.524533 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.524541 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.524543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.524784 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.525016 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:57.525017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:57.525017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:57.525132 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:57.525402 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:57.525408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:57.525564 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 06:50:57.525682 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:50:57.525876 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260806 06:50:57.526706 7560 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33465
I20260806 06:50:57.526811 7560 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:57.527017 7560 heartbeater.cc:511] Master 127.3.247.190:33465 requested a full tablet report, sending...
I20260806 06:50:57.527462 7101 ts_manager.cc:194] Registered new tserver with Master: 7a6c4371d97a46ffa046ff61fd79aa6b (127.3.247.131:32787)
I20260806 06:50:57.527958 7101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:45711
I20260806 06:50:57.534101 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999057, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:57.564706 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:57.551876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.3.247.190:33465)
0806 06:50:57.551948 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:57.551950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:57.552016 (+ 66us) server_negotiation.cc:407] Connection header received
0806 06:50:57.552098 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:57.552099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:57.552109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:57.552182 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:57.552757 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.553262 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.554124 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:57.554329 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:57.554415 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:57.561859 (+ 7444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:57.561861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:57.561862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:57.561880 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:57.563267 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.563711 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.563713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.563713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.563795 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:57.564023 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:57.564024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:57.564025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:57.564278 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:57.564374 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:57.564531 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:50:57.564601 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 06:50:57.565002 7101 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49830
I20260806 06:50:57.565517 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7163
I20260806 06:50:57.570636 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7297
I20260806 06:50:57.574765 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7431
I20260806 06:50:57.579411 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7069
2026-08-06T06:50:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1862 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7572](info): set up 2 sockets
Aug 06 06:50:57 dist-test-slave-0kls krb5kdc[7572](info): commencing operation
krb5kdc: starting...
W20260806 06:50:58.641685 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:50:58 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:50:58.758807 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41181
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:58.841202 7585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:58.841431 7585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:58.841455 7585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:58.843016 7585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:58.843075 7585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:58.843089 7585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:58.843102 7585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:58.843114 7585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:58.844775 7585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:41181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:58.845045 7585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:58.845269 7585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:58.850327 7585 server_base.cc:1034] running on GCE node
W20260806 06:50:58.850178 7590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:58.850155 7593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:58.850204 7591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:58.850726 7585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:58.851042 7585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:58.852192 7585 hybrid_clock.cc:648] HybridClock initialized: now 1785999058852184 us; error 35 us; skew 500 ppm
Aug 06 06:50:58 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999058, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:58.854801 7585 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:50:58.855198 7585 webserver.cc:460] Webserver started at http://127.3.247.190:46163/ using document root <none> and password file <none>
I20260806 06:50:58.855376 7585 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:58.855414 7585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:58.855505 7585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:58.856530 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "40b048c669c14a1381fa978914286ea3"
format_stamp: "Formatted at 2026-08-06 06:50:58 on dist-test-slave-0kls"
server_key: "4cd240382a7f29ae3c5421c7dbf732aa"
server_key_iv: "8ca1d4af2dbf5e8cdca111e7cfed5ec4"
server_key_version: "encryptionkey@0"
I20260806 06:50:58.856833 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "40b048c669c14a1381fa978914286ea3"
format_stamp: "Formatted at 2026-08-06 06:50:58 on dist-test-slave-0kls"
server_key: "4cd240382a7f29ae3c5421c7dbf732aa"
server_key_iv: "8ca1d4af2dbf5e8cdca111e7cfed5ec4"
server_key_version: "encryptionkey@0"
I20260806 06:50:58.858196 7585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:58.858901 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:58.859114 7585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:58.859225 7585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "40b048c669c14a1381fa978914286ea3"
format_stamp: "Formatted at 2026-08-06 06:50:58 on dist-test-slave-0kls"
server_key: "4cd240382a7f29ae3c5421c7dbf732aa"
server_key_iv: "8ca1d4af2dbf5e8cdca111e7cfed5ec4"
server_key_version: "encryptionkey@0"
I20260806 06:50:58.859301 7585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:58.872252 7585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:58.874817 7585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:58.874996 7585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:58.878981 7585 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:41181
I20260806 06:50:58.879025 7652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:41181 every 8 connection(s)
I20260806 06:50:58.879554 7585 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:58.880237 7653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:58.882563 7653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3: Bootstrap starting.
I20260806 06:50:58.883157 7653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:58.883445 7653 log.cc:824] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:58.883741 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7585
I20260806 06:50:58.883849 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:50:58.884043 4062 external_mini_cluster.cc:1428] Setting key 66f86a1200550384167e0bedf1dd1880
I20260806 06:50:58.884250 7653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3: No bootstrap required, opened a new log
I20260806 06:50:58.886999 7653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } }
I20260806 06:50:58.887128 7653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:58.887149 7653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b048c669c14a1381fa978914286ea3, State: Initialized, Role: FOLLOWER
I20260806 06:50:58.887287 7653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } }
I20260806 06:50:58.887358 7653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:58.887396 7653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:58.887441 7653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:58.888103 7653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } }
I20260806 06:50:58.888202 7653 leader_election.cc:302] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40b048c669c14a1381fa978914286ea3; no voters:
I20260806 06:50:58.888338 7653 leader_election.cc:288] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:58.888394 7658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:58.888540 7658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 40b048c669c14a1381fa978914286ea3, State: Running, Role: LEADER
I20260806 06:50:58.888578 7653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:58.888629 7658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } }
I20260806 06:50:58.888931 7658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40b048c669c14a1381fa978914286ea3. Latest consensus state: current_term: 1 leader_uuid: "40b048c669c14a1381fa978914286ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } } }
I20260806 06:50:58.889000 7658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:58.888939 7660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40b048c669c14a1381fa978914286ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b048c669c14a1381fa978914286ea3" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41181 } } }
I20260806 06:50:58.889148 7660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:58.889276 7665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:58.889768 7665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:58.891461 7665 catalog_manager.cc:1347] Generated new cluster ID: afc080d2553c4d1783992af523f75f0e
I20260806 06:50:58.891512 7665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:58 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:58.895624 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:58.884563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.3.247.190:41181)
0806 06:50:58.884737 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:58.884746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:58.884782 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:50:58.885512 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:58.885519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:58.885566 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:58.885674 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:58.886466 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:58.887483 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:58.888532 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:58.889169 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:58.889279 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:58.892371 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:58.892375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:58.892375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:58.892387 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:58.894133 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:58.894499 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:58.894502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:58.894502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:58.894589 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:58.894797 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:58.894798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:58.894799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:58.895030 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:58.895202 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:58.895281 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:50:58.895416 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260806 06:50:58.906014 7665 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:58.906730 7665 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:58.912002 7665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40b048c669c14a1381fa978914286ea3: Generated new TSK 0
I20260806 06:50:58.912230 7665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:50:58.942449 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:59.025537 7679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:59.025830 7679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:59.025873 7679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:59.027136 7679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:59.027215 7679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:59.027261 7679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:50:59.028855 7679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:59.029151 7679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:59.029397 7679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:59.034868 7684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:59.035074 7679 server_base.cc:1034] running on GCE node
W20260806 06:50:59.034868 7685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:59.034940 7687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:59.035436 7679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:59.035738 7679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:59.036890 7679 hybrid_clock.cc:648] HybridClock initialized: now 1785999059036876 us; error 26 us; skew 500 ppm
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:59.039572 7679 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:50:59.040023 7679 webserver.cc:460] Webserver started at http://127.3.247.129:45541/ using document root <none> and password file <none>
I20260806 06:50:59.040262 7679 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:59.040306 7679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:59.040498 7679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:59.041567 7679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "30823f71dc16437abc598a8cb2b6d51f"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "c33a5a6db1600ada6f2a81c00ccfa236"
server_key_iv: "9d4a4510de37228f1c890b40be53d13d"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.041991 7679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "30823f71dc16437abc598a8cb2b6d51f"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "c33a5a6db1600ada6f2a81c00ccfa236"
server_key_iv: "9d4a4510de37228f1c890b40be53d13d"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.043306 7679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:59.044000 7694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.044179 7679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:59.044270 7679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "30823f71dc16437abc598a8cb2b6d51f"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "c33a5a6db1600ada6f2a81c00ccfa236"
server_key_iv: "9d4a4510de37228f1c890b40be53d13d"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.044337 7679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:59.054468 7679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:59.056299 7679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:59.056457 7679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:59.056687 7679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:59.057001 7679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:59.057035 7679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.057080 7679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:59.057106 7679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.063143 7679 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:34237
I20260806 06:50:59.063210 7807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:34237 every 8 connection(s)
I20260806 06:50:59.063561 7679 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:59.067734 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7679
I20260806 06:50:59.067826 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:59.068042 4062 external_mini_cluster.cc:1428] Setting key e91070479b4a20f04500abea26e5881c
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:59.073813 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.064341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:33551 (local address 127.3.247.190:41181)
0806 06:50:59.064400 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:59.064402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:59.064592 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:50:59.065587 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:59.065589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:59.065601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:59.065655 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:59.066412 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.066928 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.068031 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.068287 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.068362 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.070745 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:59.070747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:59.070748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:59.070756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:59.072272 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.072734 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.072736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.072736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.072801 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.073075 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.073076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.073076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.073208 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:59.073288 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:59.073581 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 06:50:59.073686 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:59.074024 7810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.064307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:41181 (local address 127.3.247.129:33551)
0806 06:50:59.064455 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:59.064505 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:59.065460 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:59.065751 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:59.065759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:59.066107 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:59.066309 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.066314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.067046 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.067047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.067945 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.067948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.068381 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.068382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.068522 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.069159 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:59.069169 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:59.070639 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:59.072378 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.072387 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.072389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.072654 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.072878 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.072879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.072879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.073006 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.073277 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:59.073284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:59.073443 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 06:50:59.073560 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:50:59.073733 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260806 06:50:59.074566 7808 heartbeater.cc:347] Connected to a master server at 127.3.247.190:41181
I20260806 06:50:59.074671 7808 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:59.074889 7808 heartbeater.cc:511] Master 127.3.247.190:41181 requested a full tablet report, sending...
I20260806 06:50:59.075506 7617 ts_manager.cc:194] Registered new tserver with Master: 30823f71dc16437abc598a8cb2b6d51f (127.3.247.129:34237)
I20260806 06:50:59.076502 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:33551
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:50:59.096788 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:59.179122 7813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:59.179338 7813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:59.179363 7813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:59.180483 7813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:59.180518 7813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:59.180544 7813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:50:59.182096 7813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:59.182292 7813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:59.182487 7813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:59.187496 7821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:59.187505 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:59.187510 7819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:59.187646 7813 server_base.cc:1034] running on GCE node
I20260806 06:50:59.187937 7813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:59.188185 7813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:59.189333 7813 hybrid_clock.cc:648] HybridClock initialized: now 1785999059189321 us; error 29 us; skew 500 ppm
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:59.192139 7813 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:50:59.192687 7813 webserver.cc:460] Webserver started at http://127.3.247.130:36371/ using document root <none> and password file <none>
I20260806 06:50:59.192917 7813 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:59.192986 7813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:59.193147 7813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:59.194303 7813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "b71e2480039c43a28c9cd4549c17feaf"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "07b6064788b0bdf7973eadb7a08c64d2"
server_key_iv: "4cbeff73db5f43d0eaffbfa8564da01b"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.194677 7813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "b71e2480039c43a28c9cd4549c17feaf"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "07b6064788b0bdf7973eadb7a08c64d2"
server_key_iv: "4cbeff73db5f43d0eaffbfa8564da01b"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.196024 7813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:59.196748 7828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.196913 7813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:59.197012 7813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "b71e2480039c43a28c9cd4549c17feaf"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "07b6064788b0bdf7973eadb7a08c64d2"
server_key_iv: "4cbeff73db5f43d0eaffbfa8564da01b"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.197096 7813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:59.211406 7813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:59.213647 7813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:59.213847 7813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:59.214104 7813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:59.214433 7813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:59.214488 7813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.214536 7813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:59.214564 7813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.220585 7813 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44061
I20260806 06:50:59.220636 7941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44061 every 8 connection(s)
I20260806 06:50:59.220957 7813 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:59.222012 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7813
I20260806 06:50:59.222110 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:59.222288 4062 external_mini_cluster.cc:1428] Setting key 2d9c2c6da29a97ddbd14879d8aa64ef8
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
I20260806 06:50:59.231676 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.221700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:51649 (local address 127.3.247.190:41181)
0806 06:50:59.221812 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:59.221814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:59.222039 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:50:59.223316 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:59.223317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:59.223326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:59.223364 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:59.224241 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.224717 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.225755 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.226049 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.226134 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.228641 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:59.228643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:59.228643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:59.228652 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:59.230144 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.230628 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.230629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.230630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.230691 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.230954 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.230955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.230955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.231079 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:59.231150 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:59.231492 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 06:50:59.231579 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:50:59.231889 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.221647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:41181 (local address 127.3.247.130:51649)
0806 06:50:59.221882 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:59.221937 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:59.223211 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:59.223441 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:59.223445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:59.223862 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:59.224148 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.224154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.224809 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.224811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.225647 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.225649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.226147 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.226148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.226278 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.226822 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:59.226830 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:59.228536 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:59.230244 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.230297 (+ 53us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.230300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.230541 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.230758 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.230759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.230759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.230881 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.231151 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:59.231159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:59.231337 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 06:50:59.231464 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 06:50:59.231631 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
I20260806 06:50:59.232363 7942 heartbeater.cc:347] Connected to a master server at 127.3.247.190:41181
I20260806 06:50:59.232476 7942 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:59.232671 7942 heartbeater.cc:511] Master 127.3.247.190:41181 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
I20260806 06:50:59.233067 7617 ts_manager.cc:194] Registered new tserver with Master: b71e2480039c43a28c9cd4549c17feaf (127.3.247.130:44061)
I20260806 06:50:59.233453 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:51649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:50:59.246909 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:59.330283 7947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:59.330487 7947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:59.330518 7947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:59.331631 7947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:59.331668 7947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:59.331696 7947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:50:59.333238 7947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:59.333432 7947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:59.333622 7947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:59.338783 7953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:59.338771 7955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:59.338773 7952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:59.339246 7947 server_base.cc:1034] running on GCE node
I20260806 06:50:59.339466 7947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:59.339709 7947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:59.340898 7947 hybrid_clock.cc:648] HybridClock initialized: now 1785999059340886 us; error 31 us; skew 500 ppm
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:59.343457 7947 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:50:59.343918 7947 webserver.cc:460] Webserver started at http://127.3.247.131:36233/ using document root <none> and password file <none>
I20260806 06:50:59.344162 7947 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:59.344229 7947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:59.344377 7947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:59.345418 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "b4cee4f1d4674da6857d6934383b43b5"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "754f186e17a0dfbfa4c3f105f0db1698"
server_key_iv: "606ca5e045c65c7a9bea6e803ff09ff7"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.345774 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "b4cee4f1d4674da6857d6934383b43b5"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "754f186e17a0dfbfa4c3f105f0db1698"
server_key_iv: "606ca5e045c65c7a9bea6e803ff09ff7"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.347239 7947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:59.348038 7962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.348201 7947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:59.348295 7947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b4cee4f1d4674da6857d6934383b43b5"
format_stamp: "Formatted at 2026-08-06 06:50:59 on dist-test-slave-0kls"
server_key: "754f186e17a0dfbfa4c3f105f0db1698"
server_key_iv: "606ca5e045c65c7a9bea6e803ff09ff7"
server_key_version: "encryptionkey@0"
I20260806 06:50:59.348380 7947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:59.357901 7947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:59.360260 7947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:59.360419 7947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:59.360654 7947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:59.360970 7947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:59.361009 7947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.361050 7947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:59.361078 7947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:59.367080 7947 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:42833
I20260806 06:50:59.367120 8075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:42833 every 8 connection(s)
I20260806 06:50:59.367451 7947 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:50:59.372776 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 7947
I20260806 06:50:59.372872 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:59.373099 4062 external_mini_cluster.cc:1428] Setting key 5f6532443d8af5958ee9db2fdaf13cb2
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:59.376775 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.368175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:49451 (local address 127.3.247.190:41181)
0806 06:50:59.368239 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:59.368241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:59.368447 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:50:59.369400 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:59.369401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:59.369413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:59.369471 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:59.370186 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.370695 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.371717 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.371924 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.372004 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.374456 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:59.374457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:59.374458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:59.374464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:59.375596 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.375985 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.375986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.375986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.376048 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.376270 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.376270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.376270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.376366 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:59.376414 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:59.376654 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:50:59.376713 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:50:59.377005 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.368155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:41181 (local address 127.3.247.131:49451)
0806 06:50:59.368328 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:59.368363 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:59.369287 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:59.369558 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:59.369562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:59.369883 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:59.370086 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.370091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.370809 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.370811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.371623 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:59.371626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.372015 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.372016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:59.372145 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.372619 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:59.372627 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:59.374378 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:59.375661 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.375667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.375669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.375883 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.376105 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:59.376106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:59.376106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:59.376214 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:59.376417 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:59.376421 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:59.376548 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 06:50:59.376630 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:50:59.376745 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260806 06:50:59.377476 8076 heartbeater.cc:347] Connected to a master server at 127.3.247.190:41181
I20260806 06:50:59.377596 8076 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:59.377776 8076 heartbeater.cc:511] Master 127.3.247.190:41181 requested a full tablet report, sending...
I20260806 06:50:59.378237 7617 ts_manager.cc:194] Registered new tserver with Master: b4cee4f1d4674da6857d6934383b43b5 (127.3.247.131:42833)
I20260806 06:50:59.378654 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:49451
I20260806 06:50:59.384963 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:50:59.407877 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.399731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.3.247.129:34237)
0806 06:50:59.400134 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:59.400166 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:59.400178 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:50:59.400215 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:59.400217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:59.400232 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:59.400324 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:59.400893 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.401606 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.402419 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.402749 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.402846 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.405058 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:59.405067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:59.405067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:59.405074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:59.406702 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.407064 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.407067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.407068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.407126 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.407316 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.407317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.407318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.407453 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:59.407633 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:59.407680 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:50:59.407756 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":94,"threads_started":1}
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[7572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:50:59.415863 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:59.409166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41530 (local address 127.3.247.190:41181)
0806 06:50:59.409240 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:59.409243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:59.409252 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:59.409321 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:59.409322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:59.409333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:59.409411 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:59.409912 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.410470 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.411181 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:59.411392 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:59.411466 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:59.413704 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:59.413706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:59.413706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:59.413712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:59.414875 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.415177 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.415178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.415178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.415224 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:59.415381 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:59.415381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:59.415382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:59.415509 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:59.415581 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:59.415683 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:50:59.415737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 06:50:59.416076 7617 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41530
I20260806 06:50:59.416352 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7679
I20260806 06:50:59.421162 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7813
I20260806 06:50:59.425354 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7947
I20260806 06:50:59.429523 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 7585
2026-08-06T06:50:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1850 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[8088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[8088](info): set up 2 sockets
Aug 06 06:50:59 dist-test-slave-0kls krb5kdc[8088](info): commencing operation
krb5kdc: starting...
W20260806 06:51:00.462057 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:00 dist-test-slave-0kls krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:51:00.580449 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:37795
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:00.661846 8101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:00.662062 8101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:00.662086 8101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:00.663255 8101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:00.663297 8101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:00.663311 8101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:00.663324 8101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:00.663378 8101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:00.664873 8101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:00.665133 8101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:00.665360 8101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:00.670310 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:00.670322 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:00.670331 8109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:00.670373 8101 server_base.cc:1034] running on GCE node
I20260806 06:51:00.670759 8101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:00.671049 8101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:00.672216 8101 hybrid_clock.cc:648] HybridClock initialized: now 1785999060672202 us; error 27 us; skew 500 ppm
Aug 06 06:51:00 dist-test-slave-0kls krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999060, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:00.674717 8101 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:51:00.675078 8101 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:00.675153 8101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:00.675297 8101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:00.676340 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "551ae970f6e34571a0e5e7c3559a96ec"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "8b5f19fac04e81ae25db0c34a47821f3"
server_key_iv: "2263f6a13f3edb320e1f3b4596f4ed87"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.676703 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "551ae970f6e34571a0e5e7c3559a96ec"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "8b5f19fac04e81ae25db0c34a47821f3"
server_key_iv: "2263f6a13f3edb320e1f3b4596f4ed87"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.678014 8101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:00.678725 8115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:00.678946 8101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:00.679060 8101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "551ae970f6e34571a0e5e7c3559a96ec"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "8b5f19fac04e81ae25db0c34a47821f3"
server_key_iv: "2263f6a13f3edb320e1f3b4596f4ed87"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.679129 8101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:00.716826 8101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:00.718923 8101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:00.719115 8101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:00.723320 8101 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:37795
I20260806 06:51:00.723348 8167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:37795 every 8 connection(s)
I20260806 06:51:00.723672 8101 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:00.724637 8168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:00.725659 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8101
I20260806 06:51:00.725767 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:00.725984 4062 external_mini_cluster.cc:1428] Setting key a17533d0ea64ab840ff1261e8e520bd9
I20260806 06:51:00.728156 8168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: Bootstrap starting.
I20260806 06:51:00.728756 8168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:00.729041 8168 log.cc:824] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:00.729688 8168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: No bootstrap required, opened a new log
I20260806 06:51:00.731454 8168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } }
I20260806 06:51:00.731587 8168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:00.731611 8168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551ae970f6e34571a0e5e7c3559a96ec, State: Initialized, Role: FOLLOWER
I20260806 06:51:00.731746 8168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } }
I20260806 06:51:00.731822 8168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:00.731875 8168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:00.731923 8168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:00.732568 8168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } }
I20260806 06:51:00.732700 8168 leader_election.cc:302] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 551ae970f6e34571a0e5e7c3559a96ec; no voters:
I20260806 06:51:00.732862 8168 leader_election.cc:288] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 06 06:51:00 dist-test-slave-0kls krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:00.732933 8173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:00.733121 8168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:00.733109 8173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [term 1 LEADER]: Becoming Leader. State: Replica: 551ae970f6e34571a0e5e7c3559a96ec, State: Running, Role: LEADER
I20260806 06:51:00.734706 8173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } }
I20260806 06:51:00.735296 8175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "551ae970f6e34571a0e5e7c3559a96ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } } }
I20260806 06:51:00.735384 8175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:00.735575 8174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 551ae970f6e34571a0e5e7c3559a96ec. Latest consensus state: current_term: 1 leader_uuid: "551ae970f6e34571a0e5e7c3559a96ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "551ae970f6e34571a0e5e7c3559a96ec" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37795 } } }
I20260806 06:51:00.736361 8174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec [sys.catalog]: This master's current role is: LEADER
W20260806 06:51:00.736073 8187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:51:00.736424 8187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:51:00.736497 8188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:00.736678 8188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:00.737548 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:00.726527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.3.247.190:37795)
0806 06:51:00.726708 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:00.726716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:00.726743 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:51:00.727171 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:00.727178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:00.727227 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:00.727351 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:00.728133 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.729345 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.730217 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.730654 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.730775 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:00.733508 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:00.733512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:00.733512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:00.733523 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:00.735523 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:00.735990 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:00.735993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:00.735993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:00.736097 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:00.736316 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:00.736317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:00.736317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:00.736640 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:00.736786 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:00.737269 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 06:51:00.737406 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260806 06:51:00.738420 8188 catalog_manager.cc:1347] Generated new cluster ID: 06042640a1744a4ea135425f8d20828d
I20260806 06:51:00.738469 8188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:00.757155 8188 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:00.757928 8188 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:00.762816 8188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: Generated new TSK 0
I20260806 06:51:00.763041 8188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:51:00.794838 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:00.876386 8194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:00.876609 8194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:00.876636 8194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:00.877893 8194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:00.877938 8194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:00.877966 8194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:00.879478 8194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:00.879729 8194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:00.879953 8194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:00.885013 8200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:00.885013 8202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:00.885056 8194 server_base.cc:1034] running on GCE node
W20260806 06:51:00.885043 8199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:00.885608 8194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:00.885872 8194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:00.887032 8194 hybrid_clock.cc:648] HybridClock initialized: now 1785999060887030 us; error 36 us; skew 500 ppm
Aug 06 06:51:00 dist-test-slave-0kls krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999060, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:00.889549 8194 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:51:00.889951 8194 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:00.890026 8194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:00.890172 8194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:00.891227 8194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "108a50bff1ce4279acb4e31f70716062"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "90778bdb03fbb0e79127a8952043fde1"
server_key_iv: "8b518df96d463809b8a87ba54a6138fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.891607 8194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "108a50bff1ce4279acb4e31f70716062"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "90778bdb03fbb0e79127a8952043fde1"
server_key_iv: "8b518df96d463809b8a87ba54a6138fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.892921 8194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:00.893646 8208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:00.893862 8194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:00.893959 8194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "108a50bff1ce4279acb4e31f70716062"
format_stamp: "Formatted at 2026-08-06 06:51:00 on dist-test-slave-0kls"
server_key: "90778bdb03fbb0e79127a8952043fde1"
server_key_iv: "8b518df96d463809b8a87ba54a6138fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:00.894047 8194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:00.922801 8194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:00.925256 8194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:00.925431 8194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:00.925686 8194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:00.926040 8194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:00.926100 8194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:00.926144 8194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:00.926169 8194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:00.931855 8194 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:39709
I20260806 06:51:00.931932 8321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:39709 every 8 connection(s)
I20260806 06:51:00.932209 8194 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:51:00 dist-test-slave-0kls krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999060, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:00.940119 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8194
I20260806 06:51:00.940245 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:00.940424 4062 external_mini_cluster.cc:1428] Setting key ba5da1f129d19acdbb0d82bf0a69d7cb
I20260806 06:51:00.941546 8324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:00.932862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37795 (local address 127.3.247.129:55889)
0806 06:51:00.933019 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:00.933071 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:00.933616 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:00.933953 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:00.933956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:00.934222 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:00.934752 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:00.934757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.935480 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.935481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:00.936265 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:00.936267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.936655 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.936656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:00.936772 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:00.937217 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:00.937225 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:00.938588 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:00.940092 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:00.940099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:00.940101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:00.940397 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:00.940613 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:00.940614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:00.940614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:00.940725 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:00.940998 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:00.941005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:00.941166 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 06:51:00.941265 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:51:00.941387 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260806 06:51:00.941569 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:00.932913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:55889 (local address 127.3.247.190:37795)
0806 06:51:00.932956 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:00.932959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:00.933156 (+ 197us) server_negotiation.cc:407] Connection header received
0806 06:51:00.933703 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:00.933704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:00.933745 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:00.933862 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:00.934875 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.935378 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.936365 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:00.936576 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:00.936652 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:00.938747 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:00.938748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:00.938748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:00.938755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:00.939997 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:00.940471 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:00.940472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:00.940473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:00.940533 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:00.940806 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:00.940807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:00.940807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:00.940925 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:00.941010 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:00.941316 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 06:51:00.941417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:00.942276 8322 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37795
I20260806 06:51:00.942384 8322 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:00.942569 8322 heartbeater.cc:511] Master 127.3.247.190:37795 requested a full tablet report, sending...
I20260806 06:51:00.943109 8132 ts_manager.cc:194] Registered new tserver with Master: 108a50bff1ce4279acb4e31f70716062 (127.3.247.129:39709)
I20260806 06:51:00.943746 8132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:55889
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:51:00.965565 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:01.048277 8327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.048534 8327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.048578 8327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.049755 8327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.049863 8327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.049916 8327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:01.051448 8327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.051739 8327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.051977 8327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:01.057016 8332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.057019 8335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.057160 8327 server_base.cc:1034] running on GCE node
W20260806 06:51:01.057019 8333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.057592 8327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.057901 8327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.059062 8327 hybrid_clock.cc:648] HybridClock initialized: now 1785999061059053 us; error 40 us; skew 500 ppm
Aug 06 06:51:01 dist-test-slave-0kls krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999061, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:01.061583 8327 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:51:01.061941 8327 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.061991 8327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.062085 8327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.063079 8327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "a4f85f8623184ca8acc0c870b0aa5980"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "38776711ea7212b973975d98859375da"
server_key_iv: "7cda7029449703ab2134136fbd68cab4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.063385 8327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "a4f85f8623184ca8acc0c870b0aa5980"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "38776711ea7212b973975d98859375da"
server_key_iv: "7cda7029449703ab2134136fbd68cab4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.064566 8327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:01.065217 8341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.065382 8327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:01.065471 8327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "a4f85f8623184ca8acc0c870b0aa5980"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "38776711ea7212b973975d98859375da"
server_key_iv: "7cda7029449703ab2134136fbd68cab4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.065533 8327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.098860 8327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.101253 8327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.101370 8327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.101564 8327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:01.101918 8327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:01.101955 8327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.102001 8327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:01.102030 8327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.107585 8327 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:33729
I20260806 06:51:01.107772 8454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:33729 every 8 connection(s)
I20260806 06:51:01.107959 8327 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:01.110711 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8327
I20260806 06:51:01.110816 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:01.110997 4062 external_mini_cluster.cc:1428] Setting key 125d4d3bc058389359bd77b2afb95ff0
Aug 06 06:51:01 dist-test-slave-0kls krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999061, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:01.118122 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.108719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:52447 (local address 127.3.247.190:37795)
0806 06:51:01.108770 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.108772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.109000 (+ 228us) server_negotiation.cc:407] Connection header received
0806 06:51:01.109527 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.109528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.109541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.109595 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.110557 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.111275 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.112380 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.112606 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.112683 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.115237 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.115239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.115239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:01.115247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.116677 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.117101 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.117103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.117103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.117166 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.117415 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.117416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.117417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.117535 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.117615 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.117920 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.118021 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:51:01.118278 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.108696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37795 (local address 127.3.247.130:52447)
0806 06:51:01.108874 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.108910 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.109410 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.109666 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.109670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.109948 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.110430 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.110436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.111394 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.111395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.112277 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.112279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.112700 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.112701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.112868 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.113405 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:01.113413 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.115129 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.116778 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:01.116785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:01.116787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:01.117008 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:01.117234 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:01.117235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:01.117235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:01.117346 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:01.117604 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.117610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.117752 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.117885 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.118044 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
I20260806 06:51:01.118844 8455 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37795
I20260806 06:51:01.118948 8455 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:01.119096 8455 heartbeater.cc:511] Master 127.3.247.190:37795 requested a full tablet report, sending...
I20260806 06:51:01.119484 8132 ts_manager.cc:194] Registered new tserver with Master: a4f85f8623184ca8acc0c870b0aa5980 (127.3.247.130:33729)
I20260806 06:51:01.119837 8132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:52447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:51:01.136215 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:01.217008 8460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.217216 8460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.217242 8460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.218369 8460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.218405 8460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.218430 8460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:01.219838 8460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.220028 8460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.220203 8460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:01.225234 8468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.225309 8465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.225313 8466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.225445 8460 server_base.cc:1034] running on GCE node
I20260806 06:51:01.225718 8460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.225998 8460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.227139 8460 hybrid_clock.cc:648] HybridClock initialized: now 1785999061227115 us; error 40 us; skew 500 ppm
Aug 06 06:51:01 dist-test-slave-0kls krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999061, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:01.229652 8460 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:51:01.230091 8460 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.230166 8460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.230319 8460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.231396 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "69d9c537f8f547719a3004678036856f"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "a69b85dd58926e44923c68ad9478b9ac"
server_key_iv: "c8682461e25a749cf0c729c28ab482e4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.231756 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "69d9c537f8f547719a3004678036856f"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "a69b85dd58926e44923c68ad9478b9ac"
server_key_iv: "c8682461e25a749cf0c729c28ab482e4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.233026 8460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:01.233835 8474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.234006 8460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:01.234105 8460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "69d9c537f8f547719a3004678036856f"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "a69b85dd58926e44923c68ad9478b9ac"
server_key_iv: "c8682461e25a749cf0c729c28ab482e4"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.234187 8460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.259832 8460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.261731 8460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.261927 8460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.262192 8460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:01.262522 8460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:01.262578 8460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.262615 8460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:01.262645 8460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.268666 8460 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:37935
I20260806 06:51:01.268714 8587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:37935 every 8 connection(s)
I20260806 06:51:01.268998 8460 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:01.271252 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8460
I20260806 06:51:01.271363 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:01.271576 4062 external_mini_cluster.cc:1428] Setting key 8cb1aff772b8446eb8164287be529386
Aug 06 06:51:01 dist-test-slave-0kls krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999061, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:01.278657 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.269764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:57107 (local address 127.3.247.190:37795)
0806 06:51:01.269844 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.269847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.270047 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:51:01.270581 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.270582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.270595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.270652 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.271833 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.272418 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.273408 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.273587 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.273638 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.275682 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.275684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.275684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:01.275692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.277117 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.277561 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.277562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.277563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.277625 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.277962 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.277963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.277963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.278110 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.278191 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.278479 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.278562 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:51:01.278860 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.269747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37795 (local address 127.3.247.131:57107)
0806 06:51:01.269920 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.269959 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.270469 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.270726 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.270730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.270994 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.271711 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.271719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.272521 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.272522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.273337 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.273339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.273658 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.273659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.273756 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.274242 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:01.274250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.275583 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.277222 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:01.277230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:01.277232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:01.277462 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:01.277693 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:01.277694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:01.277694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:01.277885 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:01.278182 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.278188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.278336 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.278442 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.278619 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260806 06:51:01.279436 8588 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37795
I20260806 06:51:01.279549 8588 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:01.279706 8588 heartbeater.cc:511] Master 127.3.247.190:37795 requested a full tablet report, sending...
I20260806 06:51:01.280124 8132 ts_manager.cc:194] Registered new tserver with Master: 69d9c537f8f547719a3004678036856f (127.3.247.131:37935)
I20260806 06:51:01.280503 8132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:57107
I20260806 06:51:01.283252 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:51:01.290318 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.284521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.3.247.190:37795)
0806 06:51:01.284586 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.284589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.284680 (+ 91us) server_negotiation.cc:407] Connection header received
0806 06:51:01.284759 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.284761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.284770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.284825 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.285376 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.285918 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.286628 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.286836 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.286922 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.288011 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.288012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.288012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:01.288019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.289118 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.289458 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.289459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.289459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.289514 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:01.289715 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:01.289716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:01.289716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:01.289913 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.290023 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.290134 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.290214 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:51:01.291711 8132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:01.293224 8132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:01.300046 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.3.247.130:33729)
0806 06:51:01.296262 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.296267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.296278 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:01.296314 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.296316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.296331 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.296410 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:01.296767 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297828 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299108 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299609 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299710 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299773 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.299915 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.299955 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":82,"threads_started":1}
I20260806 06:51:01.300065 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:33729 (local address 127.0.0.1:56546)
0806 06:51:01.295623 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.295668 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.295738 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.296655 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.296655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.296657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:01.296697 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.296698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297923 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.297924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.299043 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.299045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299712 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.299797 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299813 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.299884 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.299943 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260806 06:51:01.300233 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.3.247.129:39709)
0806 06:51:01.295895 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.295901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.296612 (+ 711us) server_negotiation.cc:407] Connection header received
0806 06:51:01.296654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.296656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.296669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.296753 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:01.296881 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297698 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.298896 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299639 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299714 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299775 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.300109 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.300149 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260806 06:51:01.300233 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.3.247.131:37935)
0806 06:51:01.295895 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.295901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.295911 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:01.295972 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.295974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.295991 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.296077 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:01.296235 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297462 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299057 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299730 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299789 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299834 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.300109 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.300152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260806 06:51:01.300402 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39709 (local address 127.0.0.1:59234)
0806 06:51:01.296522 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.296544 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.296598 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.296763 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.296764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.296767 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:01.296817 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.296819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297824 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.297825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.298804 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.298807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299811 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.299876 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299880 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.299926 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.299978 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260806 06:51:01.300572 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.295472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37935 (local address 127.0.0.1:60478)
0806 06:51:01.295816 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.295825 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.295861 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.296099 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.296101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.296103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:01.296155 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.296156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.297569 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.297570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.298956 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.298959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.299862 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.299863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.299922 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.299926 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.299969 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.300005 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":99,"threads_started":1}
I20260806 06:51:01.301112 8389 tablet_service.cc:1467] Processing CreateTablet for tablet db7101a58aa7401abde567fa28ca6be0 (DEFAULT_TABLE table=test-table [id=dd2f7d55a10a4f648dcef3afd27c96ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.301163 8256 tablet_service.cc:1467] Processing CreateTablet for tablet db7101a58aa7401abde567fa28ca6be0 (DEFAULT_TABLE table=test-table [id=dd2f7d55a10a4f648dcef3afd27c96ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.301456 8389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7101a58aa7401abde567fa28ca6be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.301473 8256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7101a58aa7401abde567fa28ca6be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.302461 8522 tablet_service.cc:1467] Processing CreateTablet for tablet db7101a58aa7401abde567fa28ca6be0 (DEFAULT_TABLE table=test-table [id=dd2f7d55a10a4f648dcef3afd27c96ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.302790 8522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7101a58aa7401abde567fa28ca6be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.304399 8605 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Bootstrap starting.
I20260806 06:51:01.304854 8606 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: Bootstrap starting.
I20260806 06:51:01.305217 8605 tablet_bootstrap.cc:654] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.305560 8606 tablet_bootstrap.cc:654] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.305896 8607 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Bootstrap starting.
I20260806 06:51:01.305946 8605 log.cc:824] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.306250 8606 log.cc:824] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.306629 8607 tablet_bootstrap.cc:654] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.306747 8605 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: No bootstrap required, opened a new log
I20260806 06:51:01.306813 8605 ts_tablet_manager.cc:1397] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:51:01.307062 8606 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: No bootstrap required, opened a new log
I20260806 06:51:01.307108 8606 ts_tablet_manager.cc:1397] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:51:01.307225 8607 log.cc:824] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.308000 8607 tablet_bootstrap.cc:492] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: No bootstrap required, opened a new log
I20260806 06:51:01.308053 8607 ts_tablet_manager.cc:1397] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:51:01.308594 8606 raft_consensus.cc:348] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.308717 8606 raft_consensus.cc:374] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.308737 8606 raft_consensus.cc:729] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108a50bff1ce4279acb4e31f70716062, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.308861 8606 consensus_queue.cc:260] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.308867 8605 raft_consensus.cc:348] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.309031 8605 raft_consensus.cc:374] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.309069 8605 raft_consensus.cc:729] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f85f8623184ca8acc0c870b0aa5980, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.309139 8322 heartbeater.cc:503] Master 127.3.247.190:37795 was elected leader, sending a full tablet report...
I20260806 06:51:01.309135 8606 ts_tablet_manager.cc:1428] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:51:01.309178 8605 consensus_queue.cc:260] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.309456 8605 ts_tablet_manager.cc:1428] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:01.309486 8455 heartbeater.cc:503] Master 127.3.247.190:37795 was elected leader, sending a full tablet report...
I20260806 06:51:01.310348 8607 raft_consensus.cc:348] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.310523 8607 raft_consensus.cc:374] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.310573 8607 raft_consensus.cc:729] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d9c537f8f547719a3004678036856f, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.310686 8607 consensus_queue.cc:260] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.310973 8607 ts_tablet_manager.cc:1428] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:01.311003 8588 heartbeater.cc:503] Master 127.3.247.190:37795 was elected leader, sending a full tablet report...
W20260806 06:51:01.358578 8456 tablet.cc:2369] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:01.383215 8613 raft_consensus.cc:482] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:01.383428 8613 raft_consensus.cc:504] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.383823 8613 leader_election.cc:288] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4f85f8623184ca8acc0c870b0aa5980 (127.3.247.130:33729), 108a50bff1ce4279acb4e31f70716062 (127.3.247.129:39709)
I20260806 06:51:01.387842 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.384004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:33729 (local address 127.3.247.131:47563)
0806 06:51:01.384091 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.384103 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.384151 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.384323 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.384325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.384326 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:01.384431 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.384433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.385377 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.385378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.386745 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.386748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.387537 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.387538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.387627 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.387630 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.387684 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.387737 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":78}
I20260806 06:51:01.387859 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.384124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:43661 (local address 127.3.247.129:39709)
0806 06:51:01.384167 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.384169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.384322 (+ 153us) server_negotiation.cc:407] Connection header received
0806 06:51:01.384365 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.384366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.384376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.384423 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:01.384574 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.385285 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.386898 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.387467 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.387538 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.387546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.387740 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.387786 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:01.387859 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.384077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:47563 (local address 127.3.247.130:33729)
0806 06:51:01.384147 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.384149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.384160 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:01.384233 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.384234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.384241 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.384319 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:01.384496 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.385286 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.386849 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.387424 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.387506 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.387517 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.387740 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.387786 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:51:01.387851 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.384050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39709 (local address 127.3.247.131:43661)
0806 06:51:01.384239 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.384247 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.384307 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.384437 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.384438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.384440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:01.384503 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.384504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.385378 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.385378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.386745 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:01.386748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.387537 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.387538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:01.387617 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.387622 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.387684 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.387737 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":64,"thread_start_us":116,"threads_started":1}
I20260806 06:51:01.388285 8409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db7101a58aa7401abde567fa28ca6be0" candidate_uuid: "69d9c537f8f547719a3004678036856f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f85f8623184ca8acc0c870b0aa5980" is_pre_election: true
I20260806 06:51:01.388315 8276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db7101a58aa7401abde567fa28ca6be0" candidate_uuid: "69d9c537f8f547719a3004678036856f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108a50bff1ce4279acb4e31f70716062" is_pre_election: true
I20260806 06:51:01.388473 8409 raft_consensus.cc:2454] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d9c537f8f547719a3004678036856f in term 0.
I20260806 06:51:01.388477 8276 raft_consensus.cc:2454] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d9c537f8f547719a3004678036856f in term 0.
I20260806 06:51:01.388715 8475 leader_election.cc:302] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108a50bff1ce4279acb4e31f70716062, 69d9c537f8f547719a3004678036856f; no voters:
I20260806 06:51:01.388908 8613 raft_consensus.cc:2781] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:01.388947 8613 raft_consensus.cc:482] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:01.388962 8613 raft_consensus.cc:3037] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:01.389607 8613 raft_consensus.cc:504] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.389752 8613 leader_election.cc:288] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [CANDIDATE]: Term 1 election: Requested vote from peers a4f85f8623184ca8acc0c870b0aa5980 (127.3.247.130:33729), 108a50bff1ce4279acb4e31f70716062 (127.3.247.129:39709)
I20260806 06:51:01.389906 8409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db7101a58aa7401abde567fa28ca6be0" candidate_uuid: "69d9c537f8f547719a3004678036856f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f85f8623184ca8acc0c870b0aa5980"
I20260806 06:51:01.389968 8276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db7101a58aa7401abde567fa28ca6be0" candidate_uuid: "69d9c537f8f547719a3004678036856f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108a50bff1ce4279acb4e31f70716062"
I20260806 06:51:01.389986 8409 raft_consensus.cc:3037] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:01.390048 8276 raft_consensus.cc:3037] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:01.390851 8276 raft_consensus.cc:2454] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d9c537f8f547719a3004678036856f in term 1.
I20260806 06:51:01.390851 8409 raft_consensus.cc:2454] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d9c537f8f547719a3004678036856f in term 1.
I20260806 06:51:01.391052 8478 leader_election.cc:302] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69d9c537f8f547719a3004678036856f, a4f85f8623184ca8acc0c870b0aa5980; no voters:
I20260806 06:51:01.391144 8613 raft_consensus.cc:2781] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:01.391297 8613 raft_consensus.cc:686] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 1 LEADER]: Becoming Leader. State: Replica: 69d9c537f8f547719a3004678036856f, State: Running, Role: LEADER
I20260806 06:51:01.391393 8613 consensus_queue.cc:237] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } }
I20260806 06:51:01.392290 8131 catalog_manager.cc:5495] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f reported cstate change: term changed from 0 to 1, leader changed from <none> to 69d9c537f8f547719a3004678036856f (127.3.247.131). New cstate: current_term: 1 leader_uuid: "69d9c537f8f547719a3004678036856f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69d9c537f8f547719a3004678036856f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 } health_report { overall_health: UNKNOWN } } }
I20260806 06:51:01.414919 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.412321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60494 (local address 127.3.247.131:37935)
0806 06:51:01.412386 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.412388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.412409 (+ 21us) server_negotiation.cc:407] Connection header received
0806 06:51:01.412454 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.412454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.412473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.412551 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:01.412696 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.413173 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.414073 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:01.414308 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:01.414372 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:01.414560 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:01.414674 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:01.414773 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.414793 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.414812 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:51:01.416708 8276 raft_consensus.cc:1261] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 1 FOLLOWER]: Refusing update from remote peer 69d9c537f8f547719a3004678036856f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:01.416709 8409 raft_consensus.cc:1261] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 1 FOLLOWER]: Refusing update from remote peer 69d9c537f8f547719a3004678036856f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:01.417043 8613 consensus_queue.cc:1035] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4f85f8623184ca8acc0c870b0aa5980" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:01.417199 8615 consensus_queue.cc:1035] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [LEADER]: Connected to new peer: Peer: permanent_uuid: "108a50bff1ce4279acb4e31f70716062" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:01.419385 8618 mvcc.cc:205] Tried to move back new op lower bound from 7315452155560185856 to 7315452155459997696. Current Snapshot: MvccSnapshot[applied={T|T < 7315452155560185856}]
I20260806 06:51:01.420266 8619 mvcc.cc:205] Tried to move back new op lower bound from 7315452155560185856 to 7315452155459997696. Current Snapshot: MvccSnapshot[applied={T|T < 7315452155560185856}]
I20260806 06:51:01.421307 8131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:01.421406 8131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:01.422201 8131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 551ae970f6e34571a0e5e7c3559a96ec: Sending DeleteTablet for 3 replicas of tablet db7101a58aa7401abde567fa28ca6be0
I20260806 06:51:01.422614 8256 tablet_service.cc:1514] Processing DeleteTablet for tablet db7101a58aa7401abde567fa28ca6be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:01 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:59234
I20260806 06:51:01.422614 8522 tablet_service.cc:1514] Processing DeleteTablet for tablet db7101a58aa7401abde567fa28ca6be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:01 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:60478
I20260806 06:51:01.422628 8389 tablet_service.cc:1514] Processing DeleteTablet for tablet db7101a58aa7401abde567fa28ca6be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:01 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56546
I20260806 06:51:01.422868 8630 tablet_replica.cc:330] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: stopping tablet replica
I20260806 06:51:01.422926 8628 tablet_replica.cc:330] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062: stopping tablet replica
I20260806 06:51:01.422967 8630 raft_consensus.cc:2229] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:01.423015 8628 raft_consensus.cc:2229] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:01.423061 8630 raft_consensus.cc:2258] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:01.423094 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8194
I20260806 06:51:01.423121 8628 raft_consensus.cc:2258] T db7101a58aa7401abde567fa28ca6be0 P 108a50bff1ce4279acb4e31f70716062 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:01.423623 8629 tablet_replica.cc:330] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: stopping tablet replica
I20260806 06:51:01.423710 8629 raft_consensus.cc:2229] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:01.423801 8629 raft_consensus.cc:2258] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:01.424058 8630 ts_tablet_manager.cc:1905] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:01.424240 8629 ts_tablet_manager.cc:1905] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:01.425901 8630 ts_tablet_manager.cc:1918] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:01.425964 8630 log.cc:1196] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/db7101a58aa7401abde567fa28ca6be0
I20260806 06:51:01.426225 8630 ts_tablet_manager.cc:1939] T db7101a58aa7401abde567fa28ca6be0 P a4f85f8623184ca8acc0c870b0aa5980: Deleting consensus metadata
I20260806 06:51:01.426273 8629 ts_tablet_manager.cc:1918] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:01.426337 8629 log.cc:1196] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/db7101a58aa7401abde567fa28ca6be0
I20260806 06:51:01.426590 8629 ts_tablet_manager.cc:1939] T db7101a58aa7401abde567fa28ca6be0 P 69d9c537f8f547719a3004678036856f: Deleting consensus metadata
I20260806 06:51:01.426610 8119 catalog_manager.cc:4841] TS a4f85f8623184ca8acc0c870b0aa5980 (127.3.247.130:33729): tablet db7101a58aa7401abde567fa28ca6be0 (table test-table [id=dd2f7d55a10a4f648dcef3afd27c96ca]) successfully deleted
I20260806 06:51:01.426960 8118 catalog_manager.cc:4841] TS 69d9c537f8f547719a3004678036856f (127.3.247.131:37935): tablet db7101a58aa7401abde567fa28ca6be0 (table test-table [id=dd2f7d55a10a4f648dcef3afd27c96ca]) successfully deleted
W20260806 06:51:01.430122 8116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:39709 (error 108)
I20260806 06:51:01.430416 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8327
I20260806 06:51:01.430603 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.430482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39709 (local address 127.0.0.1:59244)
0806 06:51:01.430529 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.430565 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:51:01.431120 8116 catalog_manager.cc:4584] TS 108a50bff1ce4279acb4e31f70716062 (127.3.247.129:39709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db7101a58aa7401abde567fa28ca6be0: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39709: connect: Connection refused (error 111)
I20260806 06:51:01.435583 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8460
I20260806 06:51:01.440456 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8101
2026-08-06T06:51:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2010 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T06:51:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:01Z Disabled control of system clock
I20260806 06:51:01.467662 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:42739
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:42739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:51:01.546733 8635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.546939 8635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.546960 8635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.548044 8635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:01.548080 8635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.548094 8635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.548106 8635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:01.548120 8635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:01.549364 8635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:42739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.549548 8635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.549783 8635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:01.554757 8641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.554723 8640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.554742 8643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.554780 8635 server_base.cc:1034] running on GCE node
I20260806 06:51:01.555192 8635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.555511 8635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.556666 8635 hybrid_clock.cc:648] HybridClock initialized: now 1785999061556648 us; error 38 us; skew 500 ppm
I20260806 06:51:01.557927 8635 webserver.cc:460] Webserver started at http://127.3.247.190:42245/ using document root <none> and password file <none>
I20260806 06:51:01.558184 8635 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.558255 8635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.558413 8635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.559476 8635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "145855f695944ebb8d4fd27ffe1b1491"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "b2adfcc3b06f2d5fc7ce191fb97fd283"
server_key_iv: "0b75dd859f73e83d3a43c6d0eccfea03"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.559875 8635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "145855f695944ebb8d4fd27ffe1b1491"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "b2adfcc3b06f2d5fc7ce191fb97fd283"
server_key_iv: "0b75dd859f73e83d3a43c6d0eccfea03"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.561277 8635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:01.562001 8649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.562211 8635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:01.562282 8635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "145855f695944ebb8d4fd27ffe1b1491"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "b2adfcc3b06f2d5fc7ce191fb97fd283"
server_key_iv: "0b75dd859f73e83d3a43c6d0eccfea03"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.562333 8635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.565526 8635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.565753 8635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.565907 8635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.569499 8635 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:42739
I20260806 06:51:01.569522 8701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:42739 every 8 connection(s)
I20260806 06:51:01.569924 8635 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:01.570611 8702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.571417 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8635
I20260806 06:51:01.571532 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:01.571717 4062 external_mini_cluster.cc:1428] Setting key 9887d6e99a450775ede433359355f8a9
I20260806 06:51:01.574014 8702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: Bootstrap starting.
I20260806 06:51:01.574244 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.572250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.3.247.190:42739)
0806 06:51:01.572446 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.572453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.572477 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:51:01.572901 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.572907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.572946 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.573063 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.573735 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.573738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.573739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.573742 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.573830 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.573845 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.573911 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.573985 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.574056 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260806 06:51:01.574725 8702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.575064 8702 log.cc:824] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.575695 8702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: No bootstrap required, opened a new log
I20260806 06:51:01.577019 8702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } }
I20260806 06:51:01.577181 8702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.577226 8702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145855f695944ebb8d4fd27ffe1b1491, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.577347 8702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } }
I20260806 06:51:01.577454 8702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:01.577498 8702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:01.577556 8702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:01.578161 8702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } }
I20260806 06:51:01.578281 8702 leader_election.cc:302] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 145855f695944ebb8d4fd27ffe1b1491; no voters:
I20260806 06:51:01.578449 8702 leader_election.cc:288] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:01.578763 8707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:01.578791 8702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:01.578912 8707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [term 1 LEADER]: Becoming Leader. State: Replica: 145855f695944ebb8d4fd27ffe1b1491, State: Running, Role: LEADER
I20260806 06:51:01.579049 8707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } }
I20260806 06:51:01.579416 8709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 145855f695944ebb8d4fd27ffe1b1491. Latest consensus state: current_term: 1 leader_uuid: "145855f695944ebb8d4fd27ffe1b1491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } } }
I20260806 06:51:01.579427 8708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "145855f695944ebb8d4fd27ffe1b1491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145855f695944ebb8d4fd27ffe1b1491" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42739 } } }
I20260806 06:51:01.579566 8709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:01.579571 8708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:01.579824 8716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:01.580328 8716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:01.581828 8716 catalog_manager.cc:1347] Generated new cluster ID: 76395cc8941a42709917e2709ab35e55
I20260806 06:51:01.581904 8716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:01.597395 8716 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:01.597602 8716 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:01.610301 8716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: Generated new TSK 0
I20260806 06:51:01.610530 8716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:01.613298 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--builtin_ntp_servers=127.3.247.148:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:51:01.694963 8726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.695204 8726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.695242 8726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.696431 8726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.696497 8726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.696543 8726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:01.698058 8726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.698325 8726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.698639 8726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:01.704602 8731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.704630 8732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.704857 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.704883 8726 server_base.cc:1034] running on GCE node
I20260806 06:51:01.705111 8726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.705361 8726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.706532 8726 hybrid_clock.cc:648] HybridClock initialized: now 1785999061706517 us; error 27 us; skew 500 ppm
I20260806 06:51:01.707751 8726 webserver.cc:460] Webserver started at http://127.3.247.129:32859/ using document root <none> and password file <none>
I20260806 06:51:01.707959 8726 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.708001 8726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.708092 8726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.709103 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "cffc90bf71f140918701bbe217a2752b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "2c4fb6b14f68b1ecbea4aaf03925b9e3"
server_key_iv: "d634c3799730a95e59dca2c4074af45c"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.709393 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "cffc90bf71f140918701bbe217a2752b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "2c4fb6b14f68b1ecbea4aaf03925b9e3"
server_key_iv: "d634c3799730a95e59dca2c4074af45c"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.710747 8726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:01.711395 8740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.711584 8726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:01.711671 8726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "cffc90bf71f140918701bbe217a2752b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "2c4fb6b14f68b1ecbea4aaf03925b9e3"
server_key_iv: "d634c3799730a95e59dca2c4074af45c"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.711727 8726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.715075 8726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.715287 8726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.715378 8726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.715528 8726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:01.715756 8726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:01.715786 8726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.715806 8726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:01.715817 8726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.721192 8726 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:39847
I20260806 06:51:01.721273 8853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:39847 every 8 connection(s)
I20260806 06:51:01.721487 8726 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:01.724721 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.722298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:32875 (local address 127.3.247.190:42739)
0806 06:51:01.722382 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.722384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.722664 (+ 280us) server_negotiation.cc:407] Connection header received
0806 06:51:01.723112 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.723113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.723120 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.723182 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.724509 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.724510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.724510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.724515 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.724533 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.724537 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.724596 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.724651 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.724664 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:51:01.724941 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.722296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42739 (local address 127.3.247.129:32875)
0806 06:51:01.722539 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.722571 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.723027 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.723245 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.723249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.723258 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.724353 (+ 1095us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.724353 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.724382 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.724410 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.724600 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.724602 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.724605 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.724657 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.724704 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260806 06:51:01.725476 8854 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42739
I20260806 06:51:01.725556 8854 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:01.725708 8854 heartbeater.cc:511] Master 127.3.247.190:42739 requested a full tablet report, sending...
I20260806 06:51:01.726233 8666 ts_manager.cc:194] Registered new tserver with Master: cffc90bf71f140918701bbe217a2752b (127.3.247.129:39847)
I20260806 06:51:01.728058 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8726
I20260806 06:51:01.728164 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:01.728333 4062 external_mini_cluster.cc:1428] Setting key 06659c9b65429bc6948e80da130f93c9
I20260806 06:51:01.729151 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--builtin_ntp_servers=127.3.247.148:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:51:01.808738 8857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.808934 8857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.808956 8857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.810133 8857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.810173 8857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.810199 8857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:01.811450 8857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.811630 8857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.811863 8857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:51:01.817306 8857 server_base.cc:1034] running on GCE node
W20260806 06:51:01.817301 8865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.817289 8863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.817281 8862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.817672 8857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.817994 8857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.819177 8857 hybrid_clock.cc:648] HybridClock initialized: now 1785999061819163 us; error 26 us; skew 500 ppm
I20260806 06:51:01.820343 8857 webserver.cc:460] Webserver started at http://127.3.247.130:34899/ using document root <none> and password file <none>
I20260806 06:51:01.820583 8857 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.820652 8857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.820782 8857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.821775 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "a4779a2168814e67922cb35d4a98440b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "7e7c8fb78d5c77a6bc92564c82226fb4"
server_key_iv: "3929bad65f56f74340867074100159fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.822219 8857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "a4779a2168814e67922cb35d4a98440b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "7e7c8fb78d5c77a6bc92564c82226fb4"
server_key_iv: "3929bad65f56f74340867074100159fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.823454 8857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:01.824139 8871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.824301 8857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:01.824394 8857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "a4779a2168814e67922cb35d4a98440b"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "7e7c8fb78d5c77a6bc92564c82226fb4"
server_key_iv: "3929bad65f56f74340867074100159fd"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.824476 8857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.827733 8857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.827986 8857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.828126 8857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.828337 8857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:01.828639 8857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:01.828707 8857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.828747 8857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:01.828774 8857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.834458 8857 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:35011
I20260806 06:51:01.834501 8984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:35011 every 8 connection(s)
I20260806 06:51:01.834807 8857 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:01.837507 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.835489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:37405 (local address 127.3.247.190:42739)
0806 06:51:01.835551 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.835553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.835762 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:51:01.836197 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.836198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.836202 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.836256 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.837304 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.837305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.837305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.837309 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.837321 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.837325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.837371 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.837431 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.837444 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:51:01.837668 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.835459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42739 (local address 127.3.247.130:37405)
0806 06:51:01.835642 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.835681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.836115 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.836314 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.836317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.836324 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.837165 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.837166 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.837194 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.837221 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.837369 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.837371 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.837374 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.837425 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.837470 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260806 06:51:01.838204 8985 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42739
I20260806 06:51:01.838310 8985 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:01.838454 8985 heartbeater.cc:511] Master 127.3.247.190:42739 requested a full tablet report, sending...
I20260806 06:51:01.838757 8666 ts_manager.cc:194] Registered new tserver with Master: a4779a2168814e67922cb35d4a98440b (127.3.247.130:35011)
I20260806 06:51:01.843492 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8857
I20260806 06:51:01.843570 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:01.843755 4062 external_mini_cluster.cc:1428] Setting key 5456a59da7765d8c96b87c66a808459e
I20260806 06:51:01.844542 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--builtin_ntp_servers=127.3.247.148:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:51:01.925287 8988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:01.925498 8988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:01.925519 8988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:01.926710 8988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:01.926750 8988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:01.926775 8988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:01.928218 8988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:01.928426 8988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:01.928750 8988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:01.933573 8993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.933557 8996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:01.933552 8994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:01.933764 8988 server_base.cc:1034] running on GCE node
I20260806 06:51:01.934044 8988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:01.934281 8988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:01.935444 8988 hybrid_clock.cc:648] HybridClock initialized: now 1785999061935422 us; error 37 us; skew 500 ppm
I20260806 06:51:01.936627 8988 webserver.cc:460] Webserver started at http://127.3.247.131:42459/ using document root <none> and password file <none>
I20260806 06:51:01.936856 8988 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:01.936949 8988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:01.937093 8988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:01.938274 8988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "cad2063cce4a491993257a43589d533e"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "6685ffcf06daf2a0742ff987b28c9db6"
server_key_iv: "5a214d888880d22bb3caa24e8887eb21"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.938639 8988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "cad2063cce4a491993257a43589d533e"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "6685ffcf06daf2a0742ff987b28c9db6"
server_key_iv: "5a214d888880d22bb3caa24e8887eb21"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.939918 8988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:51:01.940546 9002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.940721 8988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.940785 8988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "cad2063cce4a491993257a43589d533e"
format_stamp: "Formatted at 2026-08-06 06:51:01 on dist-test-slave-0kls"
server_key: "6685ffcf06daf2a0742ff987b28c9db6"
server_key_iv: "5a214d888880d22bb3caa24e8887eb21"
server_key_version: "encryptionkey@0"
I20260806 06:51:01.940833 8988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:01.944075 8988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:01.944300 8988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:01.944394 8988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:01.944547 8988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:01.944782 8988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:01.944810 8988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.944830 8988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:01.944842 8988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:01.950284 8988 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:32835
I20260806 06:51:01.950356 9115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:32835 every 8 connection(s)
I20260806 06:51:01.950608 8988 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:01.953337 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.951319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:50031 (local address 127.3.247.190:42739)
0806 06:51:01.951373 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.951375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.951615 (+ 240us) server_negotiation.cc:407] Connection header received
0806 06:51:01.952046 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.952047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.952052 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.952102 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.953128 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.953129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.953129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.953133 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.953146 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.953150 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.953195 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.953271 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.953283 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:51:01.953557 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.951312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42739 (local address 127.3.247.131:50031)
0806 06:51:01.951503 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.951534 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.951963 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.952153 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.952155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.952163 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.952996 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.952997 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.953023 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.953051 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.953206 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.953210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.953212 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.953269 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.953315 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260806 06:51:01.954028 9116 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42739
I20260806 06:51:01.954120 9116 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:01.954296 9116 heartbeater.cc:511] Master 127.3.247.190:42739 requested a full tablet report, sending...
I20260806 06:51:01.954643 8666 ts_manager.cc:194] Registered new tserver with Master: cad2063cce4a491993257a43589d533e (127.3.247.131:32835)
I20260806 06:51:01.959113 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 8988
I20260806 06:51:01.959225 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:01.959412 4062 external_mini_cluster.cc:1428] Setting key 4cafd5e52cf0d88a5e05d3ad98a6b79c
I20260806 06:51:01.960327 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:51:01.962687 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.961645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.3.247.190:42739)
0806 06:51:01.961713 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.961715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.961821 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:51:01.961929 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.961930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.961934 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.961986 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.962475 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.962476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.962476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.962479 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.962490 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.962494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.962541 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.962574 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.962584 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:51:01.964079 8666 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:01.965773 8666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:01.969110 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.967800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:32835 (local address 127.0.0.1:58830)
0806 06:51:01.967942 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.967978 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.968027 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.968204 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.968210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.968217 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.968707 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.968708 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.968724 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.968747 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.968966 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.968971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.968974 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.969033 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.969046 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260806 06:51:01.969139 9125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.967894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.3.247.131:32835)
0806 06:51:01.968049 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.968053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.968065 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:51:01.968088 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.968090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.968099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.968188 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.968826 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.968834 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.968834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.968839 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.968872 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.968902 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.968972 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.969049 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.969065 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260806 06:51:01.969743 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.968657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.3.247.129:39847)
0806 06:51:01.968839 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.968843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.968852 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:01.968875 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.968878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.968886 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.969115 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.969422 (+ 307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.969426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.969426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.969431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.969461 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.969478 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.969569 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.969634 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.969649 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260806 06:51:01.970116 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.968453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.3.247.130:35011)
0806 06:51:01.968682 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:01.968686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:01.968697 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:01.968723 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:01.968725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:01.968734 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:01.969554 (+ 820us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:01.969814 (+ 260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:01.969818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:01.969818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:01.969822 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:01.969855 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:01.969869 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:01.969943 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:01.970016 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:51:01.970031 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
I20260806 06:51:01.970110 9050 tablet_service.cc:1467] Processing CreateTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e (DEFAULT_TABLE table=test-table [id=4bf78639afa2457f9283e12c5d80ef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.970489 9050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e9c4b6f7d24412afb01618eb0c742e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.972975 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.968177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39847 (local address 127.0.0.1:50852)
0806 06:51:01.968264 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.968274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.968306 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.968984 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.968985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.968987 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.969319 (+ 332us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.969320 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.969326 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.969337 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.969568 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.969569 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.969570 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.969625 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.969637 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":56,"threads_started":1}
I20260806 06:51:01.972985 9130 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Bootstrap starting.
I20260806 06:51:01.973045 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:01.968016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:35011 (local address 127.0.0.1:41756)
0806 06:51:01.968124 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:01.968133 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:01.968163 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:01.968838 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:01.968839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:01.968842 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:01.969456 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:01.969456 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:01.969462 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:01.969474 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:01.969951 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:01.969952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:01.969952 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:51:01.970008 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:51:01.970021 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260806 06:51:01.973658 9130 tablet_bootstrap.cc:654] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.973737 8788 tablet_service.cc:1467] Processing CreateTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e (DEFAULT_TABLE table=test-table [id=4bf78639afa2457f9283e12c5d80ef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.973852 8919 tablet_service.cc:1467] Processing CreateTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e (DEFAULT_TABLE table=test-table [id=4bf78639afa2457f9283e12c5d80ef04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:01.974013 9130 log.cc:824] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.974152 8788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e9c4b6f7d24412afb01618eb0c742e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.974190 8919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e9c4b6f7d24412afb01618eb0c742e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:01.976752 9132 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: Bootstrap starting.
I20260806 06:51:01.977069 9130 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: No bootstrap required, opened a new log
I20260806 06:51:01.977144 9130 ts_tablet_manager.cc:1397] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 06:51:01.977435 9132 tablet_bootstrap.cc:654] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.977759 9132 log.cc:824] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.978052 9133 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Bootstrap starting.
I20260806 06:51:01.978561 9132 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: No bootstrap required, opened a new log
I20260806 06:51:01.978631 9132 ts_tablet_manager.cc:1397] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:01.978780 9133 tablet_bootstrap.cc:654] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:01.979101 9133 log.cc:824] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:01.979879 9130 raft_consensus.cc:348] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.980020 9130 raft_consensus.cc:374] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.980062 9130 raft_consensus.cc:729] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad2063cce4a491993257a43589d533e, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.980212 9133 tablet_bootstrap.cc:492] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: No bootstrap required, opened a new log
I20260806 06:51:01.980185 9130 consensus_queue.cc:260] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.980276 9133 ts_tablet_manager.cc:1397] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:01.980446 9130 ts_tablet_manager.cc:1428] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:51:01.980469 9116 heartbeater.cc:503] Master 127.3.247.190:42739 was elected leader, sending a full tablet report...
I20260806 06:51:01.981213 9132 raft_consensus.cc:348] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.981382 9132 raft_consensus.cc:374] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.981422 9132 raft_consensus.cc:729] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffc90bf71f140918701bbe217a2752b, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.981526 9132 consensus_queue.cc:260] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.981856 9132 ts_tablet_manager.cc:1428] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:51:01.981900 8854 heartbeater.cc:503] Master 127.3.247.190:42739 was elected leader, sending a full tablet report...
I20260806 06:51:01.983356 9133 raft_consensus.cc:348] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.983464 9133 raft_consensus.cc:374] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:01.983501 9133 raft_consensus.cc:729] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4779a2168814e67922cb35d4a98440b, State: Initialized, Role: FOLLOWER
I20260806 06:51:01.983637 9133 consensus_queue.cc:260] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:01.983882 9133 ts_tablet_manager.cc:1428] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 06:51:01.983882 8985 heartbeater.cc:503] Master 127.3.247.190:42739 was elected leader, sending a full tablet report...
W20260806 06:51:02.085310 8986 tablet.cc:2369] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:02.177426 9137 raft_consensus.cc:482] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:02.177572 9137 raft_consensus.cc:504] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:02.178038 9137 leader_election.cc:288] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cad2063cce4a491993257a43589d533e (127.3.247.131:32835), a4779a2168814e67922cb35d4a98440b (127.3.247.130:35011)
I20260806 06:51:02.179551 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.178241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:32835 (local address 127.3.247.129:47109)
0806 06:51:02.178308 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.178323 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:02.178386 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:02.178543 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:02.178551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:02.178554 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:02.179195 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:02.179195 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:02.179203 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:02.179216 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:02.179387 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:02.179387 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:02.179388 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:51:02.179440 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:02.179453 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:51:02.179554 9125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.178333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:47109 (local address 127.3.247.131:32835)
0806 06:51:02.178380 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.178382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.178394 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:51:02.178461 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.178462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.178468 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.178514 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.179283 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.179285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.179285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.179294 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.179313 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.179317 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.179375 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.179440 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.179453 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:51:02.179677 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.178468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:46075 (local address 127.3.247.130:35011)
0806 06:51:02.178512 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.178514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.178678 (+ 164us) server_negotiation.cc:407] Connection header received
0806 06:51:02.178708 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.178709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.178715 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.178783 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.179439 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.179440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.179441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.179446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.179460 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.179464 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.179517 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.179572 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.179584 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:51:02.179716 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.178375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:35011 (local address 127.3.247.129:46075)
0806 06:51:02.178570 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.178580 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:02.178628 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:02.178792 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:02.178793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:02.178795 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:02.179333 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:02.179334 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:02.179338 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:02.179345 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:02.179523 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:02.179524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:02.179524 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 06:51:02.179571 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:51:02.179583 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
I20260806 06:51:02.179885 9070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e9c4b6f7d24412afb01618eb0c742e" candidate_uuid: "cffc90bf71f140918701bbe217a2752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad2063cce4a491993257a43589d533e" is_pre_election: true
I20260806 06:51:02.180004 9070 raft_consensus.cc:2454] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffc90bf71f140918701bbe217a2752b in term 0.
I20260806 06:51:02.180248 8741 leader_election.cc:302] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cad2063cce4a491993257a43589d533e, cffc90bf71f140918701bbe217a2752b; no voters:
I20260806 06:51:02.180382 9137 raft_consensus.cc:2781] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:02.180361 8939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e9c4b6f7d24412afb01618eb0c742e" candidate_uuid: "cffc90bf71f140918701bbe217a2752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4779a2168814e67922cb35d4a98440b" is_pre_election: true
I20260806 06:51:02.180429 9137 raft_consensus.cc:482] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:02.180455 9137 raft_consensus.cc:3037] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.180509 8939 raft_consensus.cc:2454] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffc90bf71f140918701bbe217a2752b in term 0.
I20260806 06:51:02.181421 9137 raft_consensus.cc:504] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:02.181563 9137 leader_election.cc:288] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [CANDIDATE]: Term 1 election: Requested vote from peers cad2063cce4a491993257a43589d533e (127.3.247.131:32835), a4779a2168814e67922cb35d4a98440b (127.3.247.130:35011)
I20260806 06:51:02.181748 8939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e9c4b6f7d24412afb01618eb0c742e" candidate_uuid: "cffc90bf71f140918701bbe217a2752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4779a2168814e67922cb35d4a98440b"
I20260806 06:51:02.181865 8939 raft_consensus.cc:3037] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.182111 9070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e9c4b6f7d24412afb01618eb0c742e" candidate_uuid: "cffc90bf71f140918701bbe217a2752b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad2063cce4a491993257a43589d533e"
I20260806 06:51:02.182228 9070 raft_consensus.cc:3037] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.183017 8939 raft_consensus.cc:2454] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffc90bf71f140918701bbe217a2752b in term 1.
I20260806 06:51:02.183220 8741 leader_election.cc:302] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4779a2168814e67922cb35d4a98440b, cffc90bf71f140918701bbe217a2752b; no voters:
I20260806 06:51:02.183355 9137 raft_consensus.cc:2781] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:02.183562 9137 raft_consensus.cc:686] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [term 1 LEADER]: Becoming Leader. State: Replica: cffc90bf71f140918701bbe217a2752b, State: Running, Role: LEADER
I20260806 06:51:02.183681 9137 consensus_queue.cc:237] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } }
I20260806 06:51:02.183954 9070 raft_consensus.cc:2454] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffc90bf71f140918701bbe217a2752b in term 1.
I20260806 06:51:02.184615 8666 catalog_manager.cc:5495] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b reported cstate change: term changed from 0 to 1, leader changed from <none> to cffc90bf71f140918701bbe217a2752b (127.3.247.129). New cstate: current_term: 1 leader_uuid: "cffc90bf71f140918701bbe217a2752b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cffc90bf71f140918701bbe217a2752b" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39847 } health_report { overall_health: HEALTHY } } }
I20260806 06:51:02.197216 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.196279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.3.247.129:39847)
0806 06:51:02.196344 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.196346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.196354 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:02.196399 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.196401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.196416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.196471 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.196990 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.196991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.196991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.196995 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.197011 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.197015 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.197073 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.197125 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.197139 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:51:02.199169 9070 raft_consensus.cc:1261] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 1 FOLLOWER]: Refusing update from remote peer cffc90bf71f140918701bbe217a2752b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:02.199169 8939 raft_consensus.cc:1261] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 1 FOLLOWER]: Refusing update from remote peer cffc90bf71f140918701bbe217a2752b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:02.199503 9137 consensus_queue.cc:1035] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4779a2168814e67922cb35d4a98440b" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:02.199658 9140 consensus_queue.cc:1035] T 72e9c4b6f7d24412afb01618eb0c742e P cffc90bf71f140918701bbe217a2752b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cad2063cce4a491993257a43589d533e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:51:02.201085 9117 tablet.cc:2369] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:02.204402 8665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:02.204499 8665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:02.205242 8665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 145855f695944ebb8d4fd27ffe1b1491: Sending DeleteTablet for 3 replicas of tablet 72e9c4b6f7d24412afb01618eb0c742e
I20260806 06:51:02.205588 9050 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:02 UTC) from {username='slave'} at 127.0.0.1:58830
I20260806 06:51:02.205689 8919 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:02 UTC) from {username='slave'} at 127.0.0.1:41756
I20260806 06:51:02.205770 8788 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e9c4b6f7d24412afb01618eb0c742e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:02 UTC) from {username='slave'} at 127.0.0.1:50852
I20260806 06:51:02.206003 9153 tablet_replica.cc:330] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: stopping tablet replica
I20260806 06:51:02.206111 9153 raft_consensus.cc:2229] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:02.206164 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8726
I20260806 06:51:02.206221 9153 raft_consensus.cc:2258] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:02.207796 9154 tablet_replica.cc:330] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: stopping tablet replica
I20260806 06:51:02.207921 9154 raft_consensus.cc:2229] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:02.208040 9154 raft_consensus.cc:2258] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:02.208032 9153 ts_tablet_manager.cc:1905] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:02.208513 9154 ts_tablet_manager.cc:1905] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:02.209942 9154 ts_tablet_manager.cc:1918] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:02.210032 9154 log.cc:1196] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/72e9c4b6f7d24412afb01618eb0c742e
I20260806 06:51:02.210134 9153 ts_tablet_manager.cc:1918] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:02.210215 9153 log.cc:1196] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/72e9c4b6f7d24412afb01618eb0c742e
I20260806 06:51:02.210340 9154 ts_tablet_manager.cc:1939] T 72e9c4b6f7d24412afb01618eb0c742e P a4779a2168814e67922cb35d4a98440b: Deleting consensus metadata
I20260806 06:51:02.210505 9153 ts_tablet_manager.cc:1939] T 72e9c4b6f7d24412afb01618eb0c742e P cad2063cce4a491993257a43589d533e: Deleting consensus metadata
I20260806 06:51:02.210817 8650 catalog_manager.cc:4841] TS a4779a2168814e67922cb35d4a98440b (127.3.247.130:35011): tablet 72e9c4b6f7d24412afb01618eb0c742e (table test-table [id=4bf78639afa2457f9283e12c5d80ef04]) successfully deleted
I20260806 06:51:02.210924 8650 catalog_manager.cc:4841] TS cad2063cce4a491993257a43589d533e (127.3.247.131:32835): tablet 72e9c4b6f7d24412afb01618eb0c742e (table test-table [id=4bf78639afa2457f9283e12c5d80ef04]) successfully deleted
W20260806 06:51:02.212486 8650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:39847 (error 108)
I20260806 06:51:02.213001 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.212853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39847 (local address 127.0.0.1:50880)
0806 06:51:02.212912 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.212961 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:51:02.213145 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8857
W20260806 06:51:02.213155 8650 catalog_manager.cc:4584] TS cffc90bf71f140918701bbe217a2752b (127.3.247.129:39847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72e9c4b6f7d24412afb01618eb0c742e: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39847: connect: Connection refused (error 111)
I20260806 06:51:02.219992 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8988
I20260806 06:51:02.224402 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 8635
2026-08-06T06:51:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (783 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 06:51:02.243784 4062 mini_oidc.cc:152] Starting JWKS server
I20260806 06:51:02.244843 4062 webserver.cc:460] Webserver started at https://127.0.0.1:36323/ using document root <none> and password file <none>
I20260806 06:51:02.244925 4062 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:51:02.245059 4062 webserver.cc:460] Webserver started at http://127.0.0.1:45991/ using document root <none> and password file <none>
2026-08-06T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:02Z Disabled control of system clock
I20260806 06:51:02.252877 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38081
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38081
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36323/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/testchainca.cert with env {}
W20260806 06:51:02.334353 9162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:02.334571 9162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:02.334595 9162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:02.335714 9162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:02.335747 9162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:02.335759 9162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:02.335772 9162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:02.335784 9162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:51:02.335796 9162 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:51:02.335807 9162 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36323/jwks.json
I20260806 06:51:02.337381 9162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36323/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:02.337590 9162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:02.337750 9162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:02.342700 9168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:02.342694 9170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.342724 9162 server_base.cc:1034] running on GCE node
W20260806 06:51:02.342680 9167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.343218 9162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:02.343516 9162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:02.344666 9162 hybrid_clock.cc:648] HybridClock initialized: now 1785999062344651 us; error 29 us; skew 500 ppm
I20260806 06:51:02.345954 9162 webserver.cc:460] Webserver started at http://127.3.247.190:33505/ using document root <none> and password file <none>
I20260806 06:51:02.346261 9162 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:02.346310 9162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:02.346454 9162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:02.347566 9162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "c506902f68e44388b080c2e013513306"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "8fc64d74c0222c23b8c312c855ea8563"
server_key_iv: "d8126321c649ba1709e7e5945467021e"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.347955 9162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "c506902f68e44388b080c2e013513306"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "8fc64d74c0222c23b8c312c855ea8563"
server_key_iv: "d8126321c649ba1709e7e5945467021e"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.349278 9162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:02.350039 9176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:02.350219 9162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:02.350322 9162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "c506902f68e44388b080c2e013513306"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "8fc64d74c0222c23b8c312c855ea8563"
server_key_iv: "d8126321c649ba1709e7e5945467021e"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.350412 9162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:02.365463 9162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:02.365904 9162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:02.366086 9162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:02.370265 9162 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38081
I20260806 06:51:02.370306 9228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38081 every 8 connection(s)
I20260806 06:51:02.370672 9162 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:02.371622 9229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:02.374027 9229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: Bootstrap starting.
I20260806 06:51:02.374631 9229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:02.374940 9229 log.cc:824] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:02.375638 9229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: No bootstrap required, opened a new log
I20260806 06:51:02.377167 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9162
I20260806 06:51:02.377270 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:02.377456 4062 external_mini_cluster.cc:1428] Setting key a5ec675eea08060992e938e27fc0af49
I20260806 06:51:02.377499 9229 raft_consensus.cc:348] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } }
I20260806 06:51:02.377678 9229 raft_consensus.cc:374] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:02.377705 9229 raft_consensus.cc:729] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c506902f68e44388b080c2e013513306, State: Initialized, Role: FOLLOWER
I20260806 06:51:02.377899 9229 consensus_queue.cc:260] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } }
I20260806 06:51:02.377993 9229 raft_consensus.cc:388] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:02.378032 9229 raft_consensus.cc:482] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:02.378077 9229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.378628 9229 raft_consensus.cc:504] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } }
I20260806 06:51:02.378746 9229 leader_election.cc:302] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c506902f68e44388b080c2e013513306; no voters:
I20260806 06:51:02.378928 9229 leader_election.cc:288] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:02.379001 9234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:02.379148 9234 raft_consensus.cc:686] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [term 1 LEADER]: Becoming Leader. State: Replica: c506902f68e44388b080c2e013513306, State: Running, Role: LEADER
I20260806 06:51:02.379192 9229 sys_catalog.cc:564] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:02.379256 9234 consensus_queue.cc:237] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } }
I20260806 06:51:02.379601 9234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c506902f68e44388b080c2e013513306. Latest consensus state: current_term: 1 leader_uuid: "c506902f68e44388b080c2e013513306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } } }
I20260806 06:51:02.379681 9234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:02.379612 9235 sys_catalog.cc:455] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c506902f68e44388b080c2e013513306" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c506902f68e44388b080c2e013513306" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38081 } } }
I20260806 06:51:02.379721 9235 sys_catalog.cc:458] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306 [sys.catalog]: This master's current role is: LEADER
W20260806 06:51:02.381039 9250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:51:02.381098 9250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:51:02.381147 9240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:02.381318 9240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:02.382815 9240 catalog_manager.cc:1347] Generated new cluster ID: 18109a920a754d5c97fafbbd97be844a
I20260806 06:51:02.382874 9240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:02.383703 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.378020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.3.247.190:38081)
0806 06:51:02.378264 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.378274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.378309 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:51:02.378418 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.378423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.378490 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.378635 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.378821 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.380560 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.381546 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.382089 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.382210 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.383015 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.383018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.383019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.383197 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.383233 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.383244 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.383333 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.383407 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.383487 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260806 06:51:02.384560 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:51:02.390800 9240 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:02.391309 9240 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:02.397454 9240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c506902f68e44388b080c2e013513306: Generated new TSK 0
I20260806 06:51:02.397675 9240 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:51:02.419713 9259 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 06:51:02.419809 9259 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:51:02.420094 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.404443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.3.247.190:38081)
0806 06:51:02.404501 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.404503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.404590 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:51:02.404660 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.404661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.404674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.404754 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:51:02.404877 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.405665 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.406484 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.406790 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.406862 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.407023 (+ 161us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:51:02.419872 (+ 12849us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:51:02.420024 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 06:51:02.420150 9233 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 06:51:02.420413 9258 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:02.404344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38081 (local address 127.0.0.1:59228)
0806 06:51:02.404531 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.404540 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:02.404593 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:02.404764 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:02.404764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:02.404766 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:51:02.404817 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.404818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.405755 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.405756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.406406 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.406408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.406867 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.406867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.406910 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.406928 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:51:02.419994 (+ 13066us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:51:02.420105 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.247.190:38081: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.247.190:38081: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36323/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 06:51:02.420967 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9162
2026-08-06T06:51:02Z chronyd exiting
I20260806 06:51:02.456713 4062 test_util.cc:175] -----------------------------------------------
I20260806 06:51:02.456815 4062 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785999040366271-4062-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 06:51:02.457546 4062 mini_oidc.cc:152] Starting JWKS server
I20260806 06:51:02.458325 4062 webserver.cc:460] Webserver started at https://127.0.0.1:38771/ using document root <none> and password file <none>
I20260806 06:51:02.458393 4062 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:51:02.458561 4062 webserver.cc:460] Webserver started at http://127.0.0.1:42081/ using document root <none> and password file <none>
2026-08-06T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:02Z Disabled control of system clock
I20260806 06:51:02.466143 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36203
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:39277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36203
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38771/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/testchainca.cert with env {}
W20260806 06:51:02.545287 9265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:02.545495 9265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:02.545517 9265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:02.546650 9265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:02.546685 9265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:02.546698 9265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:02.546711 9265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:02.546723 9265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:51:02.546736 9265 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:51:02.546746 9265 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38771/jwks.json
I20260806 06:51:02.548169 9265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38771/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:02.548386 9265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:02.548540 9265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:02.553370 9273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:02.553486 9270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.553375 9265 server_base.cc:1034] running on GCE node
W20260806 06:51:02.553422 9271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.553828 9265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:02.554123 9265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:02.555270 9265 hybrid_clock.cc:648] HybridClock initialized: now 1785999062555260 us; error 32 us; skew 500 ppm
I20260806 06:51:02.556480 9265 webserver.cc:460] Webserver started at http://127.3.247.190:37049/ using document root <none> and password file <none>
I20260806 06:51:02.556668 9265 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:02.556710 9265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:02.556804 9265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:02.557894 9265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "4b6a9bc1f74046638429ff5317cd0871"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "e139045705b3478f478737838eb7448a"
server_key_iv: "64de227a5424e77f2224e631ef55dfb1"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.558218 9265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "4b6a9bc1f74046638429ff5317cd0871"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "e139045705b3478f478737838eb7448a"
server_key_iv: "64de227a5424e77f2224e631ef55dfb1"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.559391 9265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:02.560278 9279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:02.560427 9265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:02.560535 9265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "4b6a9bc1f74046638429ff5317cd0871"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "e139045705b3478f478737838eb7448a"
server_key_iv: "64de227a5424e77f2224e631ef55dfb1"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.560613 9265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:02.583374 9265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:02.583716 9265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:02.583873 9265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:02.587584 9265 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:36203
I20260806 06:51:02.587659 9331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:36203 every 8 connection(s)
I20260806 06:51:02.588006 9265 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:02.588922 9332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:02.590196 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9265
I20260806 06:51:02.590276 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:02.590479 4062 external_mini_cluster.cc:1428] Setting key cb132e7d2f996da56dad1da9a49d6ea0
I20260806 06:51:02.591609 9332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: Bootstrap starting.
I20260806 06:51:02.592300 9332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:02.592588 9332 log.cc:824] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:02.593334 9332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: No bootstrap required, opened a new log
I20260806 06:51:02.595659 9332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } }
I20260806 06:51:02.595800 9332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:02.595822 9332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6a9bc1f74046638429ff5317cd0871, State: Initialized, Role: FOLLOWER
I20260806 06:51:02.595896 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.590983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.3.247.190:36203)
0806 06:51:02.591156 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.591163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.591190 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:51:02.591599 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.591605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.591663 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.591814 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.592050 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.592994 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.594085 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.594468 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.594555 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.595221 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.595224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.595225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.595376 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.595408 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.595417 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.595466 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.595583 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.595666 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260806 06:51:02.595963 9332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } }
I20260806 06:51:02.596060 9332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:02.596095 9332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:02.596141 9332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.596817 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:51:02.596843 9332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } }
I20260806 06:51:02.596935 9332 leader_election.cc:302] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b6a9bc1f74046638429ff5317cd0871; no voters:
I20260806 06:51:02.597095 9332 leader_election.cc:288] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:02.597148 9337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:02.597309 9337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [term 1 LEADER]: Becoming Leader. State: Replica: 4b6a9bc1f74046638429ff5317cd0871, State: Running, Role: LEADER
I20260806 06:51:02.597324 9332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:02.597398 9337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } }
I20260806 06:51:02.597712 9338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b6a9bc1f74046638429ff5317cd0871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } } }
I20260806 06:51:02.597836 9338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:02.597699 9337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b6a9bc1f74046638429ff5317cd0871. Latest consensus state: current_term: 1 leader_uuid: "4b6a9bc1f74046638429ff5317cd0871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6a9bc1f74046638429ff5317cd0871" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36203 } } }
I20260806 06:51:02.598032 9337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871 [sys.catalog]: This master's current role is: LEADER
W20260806 06:51:02.599086 9353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:51:02.599128 9353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:51:02.599191 9344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:02.599350 9344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:02.600639 9344 catalog_manager.cc:1347] Generated new cluster ID: 0512e74935334e3998ec1b6a99f7551e
I20260806 06:51:02.600688 9344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:02.615885 9344 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:02.616523 9344 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:02.623256 9344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b6a9bc1f74046638429ff5317cd0871: Generated new TSK 0
I20260806 06:51:02.623453 9344 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:51:02.655922 9362 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 06:51:02.656016 9362 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:51:02.656306 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.634298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.3.247.190:36203)
0806 06:51:02.634342 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.634344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.634421 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:51:02.634504 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.634505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.634514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.634571 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:51:02.634704 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.635179 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.636285 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.636464 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.636511 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.636692 (+ 181us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:51:02.656092 (+ 19400us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:51:02.656252 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:51:02.656381 9335 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 06:51:02.656450 9361 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:02.634199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36203 (local address 127.0.0.1:34502)
0806 06:51:02.634362 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.634373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:02.634438 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:02.634585 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:02.634586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:02.634588 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:51:02.634645 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.634647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.635254 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.635255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.636204 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.636206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.636533 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.636541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.636601 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.636607 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:51:02.656226 (+ 19619us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:51:02.656352 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.247.190:36203: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260806 06:51:02.656834 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9265
2026-08-06T06:51:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 06:51:02.694310 4062 mini_oidc.cc:152] Starting JWKS server
I20260806 06:51:02.695017 4062 webserver.cc:460] Webserver started at https://127.0.0.1:42775/ using document root <none> and password file <none>
I20260806 06:51:02.695088 4062 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:51:02.695252 4062 webserver.cc:460] Webserver started at http://127.0.0.1:34481/ using document root <none> and password file <none>
2026-08-06T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:02Z Disabled control of system clock
I20260806 06:51:02.703079 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41495
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:41495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42775/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:51:02.782989 9368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:02.783203 9368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:02.783224 9368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:02.784297 9368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:02.784332 9368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:02.784345 9368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:02.784363 9368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:02.784374 9368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:51:02.784386 9368 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:51:02.784399 9368 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42775/jwks.json
I20260806 06:51:02.785643 9368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:41495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42775/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:02.785888 9368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:02.786134 9368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:02.790836 9373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:02.790864 9376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.791064 9368 server_base.cc:1034] running on GCE node
W20260806 06:51:02.790836 9374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:02.791388 9368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:02.791685 9368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:02.792855 9368 hybrid_clock.cc:648] HybridClock initialized: now 1785999062792839 us; error 45 us; skew 500 ppm
I20260806 06:51:02.794113 9368 webserver.cc:460] Webserver started at http://127.3.247.190:39967/ using document root <none> and password file <none>
I20260806 06:51:02.794346 9368 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:02.794415 9368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:02.794574 9368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:02.795643 9368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "1746ce10e9e8472b8bcd797b8621e52a"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "987c5ffe0a79e7e14a29bafb593d3953"
server_key_iv: "c36300c15d555f85cb93ad19aa9dc586"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.796006 9368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "1746ce10e9e8472b8bcd797b8621e52a"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "987c5ffe0a79e7e14a29bafb593d3953"
server_key_iv: "c36300c15d555f85cb93ad19aa9dc586"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.797384 9368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:02.798166 9382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:02.798332 9368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:02.798442 9368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "1746ce10e9e8472b8bcd797b8621e52a"
format_stamp: "Formatted at 2026-08-06 06:51:02 on dist-test-slave-0kls"
server_key: "987c5ffe0a79e7e14a29bafb593d3953"
server_key_iv: "c36300c15d555f85cb93ad19aa9dc586"
server_key_version: "encryptionkey@0"
I20260806 06:51:02.798517 9368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:02.805927 9368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:02.806214 9368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:02.806365 9368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:02.810334 9368 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:41495
I20260806 06:51:02.810364 9434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:41495 every 8 connection(s)
I20260806 06:51:02.810714 9368 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:02.811342 9435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:02.814034 9435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a: Bootstrap starting.
I20260806 06:51:02.814625 9435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:02.814905 9435 log.cc:824] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:02.815661 9435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a: No bootstrap required, opened a new log
I20260806 06:51:02.817179 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9368
I20260806 06:51:02.817261 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:02.817437 4062 external_mini_cluster.cc:1428] Setting key b25675d42053cdcb600390d173171379
I20260806 06:51:02.818130 9435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } }
I20260806 06:51:02.818332 9435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:02.818382 9435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1746ce10e9e8472b8bcd797b8621e52a, State: Initialized, Role: FOLLOWER
I20260806 06:51:02.818507 9435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } }
I20260806 06:51:02.818578 9435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:02.818624 9435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:02.818681 9435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:02.819489 9435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } }
I20260806 06:51:02.819595 9435 leader_election.cc:302] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1746ce10e9e8472b8bcd797b8621e52a; no voters:
I20260806 06:51:02.819746 9435 leader_election.cc:288] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:02.819802 9440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:02.819942 9440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [term 1 LEADER]: Becoming Leader. State: Replica: 1746ce10e9e8472b8bcd797b8621e52a, State: Running, Role: LEADER
I20260806 06:51:02.819993 9435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:02.820034 9440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } }
I20260806 06:51:02.820446 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1746ce10e9e8472b8bcd797b8621e52a. Latest consensus state: current_term: 1 leader_uuid: "1746ce10e9e8472b8bcd797b8621e52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } } }
I20260806 06:51:02.820446 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1746ce10e9e8472b8bcd797b8621e52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746ce10e9e8472b8bcd797b8621e52a" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41495 } } }
I20260806 06:51:02.820562 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:02.820569 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:02.820765 9449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:02.821295 9449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:02.822866 9449 catalog_manager.cc:1347] Generated new cluster ID: a6b7e088190a4640a97940422910f3eb
I20260806 06:51:02.822917 9449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:02.824242 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.818050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.3.247.190:41495)
0806 06:51:02.818291 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.818300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.818334 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:51:02.818460 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.818467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.818540 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.818698 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:02.818924 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.820687 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.821893 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.822506 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.822625 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.823502 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:02.823506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:02.823506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:02.823720 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:02.823758 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:02.823771 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:02.823834 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:02.823955 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:51:02.824048 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260806 06:51:02.825124 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:51:02.838398 9449 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:02.838936 9449 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:02.843444 9449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1746ce10e9e8472b8bcd797b8621e52a: Generated new TSK 0
I20260806 06:51:02.843667 9449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:02.860088 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:02.856235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.3.247.190:41495)
0806 06:51:02.856325 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:02.856327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:02.856433 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:51:02.856497 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:02.856498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:02.856532 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:02.856596 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:51:02.856747 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.857258 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.858430 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.858633 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.858687 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.858880 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:51:02.859841 (+ 961us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:51:02.859978 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 06:51:02.860159 9439 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 06:51:02.860277 9464 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:02.856176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:41495 (local address 127.0.0.1:35608)
0806 06:51:02.856351 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:02.856363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:02.856434 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:02.856615 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:02.856616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:02.856618 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:51:02.856674 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.856676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.857341 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.857342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.858338 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:02.858342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:02.858714 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:02.858716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:02.858775 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:02.858781 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:51:02.859950 (+ 1169us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:51:02.860056 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.247.190:41495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260806 06:51:02.860601 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9368
E20260806 06:51:02.870990 9465 webserver.cc:533] Webserver: sslize failed:
E20260806 06:51:02.871065 9465 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T06:51:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:02Z Disabled control of system clock
I20260806 06:51:02.905315 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43907
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:43243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:43907
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/testchainca.cert with env {}
W20260806 06:51:02.992177 9469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:02.992408 9469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:02.992446 9469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:02.994148 9469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:02.994215 9469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:02.994236 9469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:02.994269 9469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:02.994292 9469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:51:02.994311 9469 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:51:02.994338 9469 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 06:51:02.996476 9469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:43907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:02.996768 9469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:02.996989 9469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:03.003177 9474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:03.003166 9475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:03.003268 9469 server_base.cc:1034] running on GCE node
W20260806 06:51:03.003386 9477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:03.003643 9469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:03.004010 9469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:03.005169 9469 hybrid_clock.cc:648] HybridClock initialized: now 1785999063005153 us; error 45 us; skew 500 ppm
I20260806 06:51:03.006688 9469 webserver.cc:460] Webserver started at http://127.3.247.190:34053/ using document root <none> and password file <none>
I20260806 06:51:03.006947 9469 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:03.006999 9469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:03.007140 9469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:03.008440 9469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "64959a6e9fe847229b078aca10d1658d"
format_stamp: "Formatted at 2026-08-06 06:51:03 on dist-test-slave-0kls"
server_key: "ac59a1e1ef4b9cc2d023d7b421f6cb5e"
server_key_iv: "28692c1d15a312e6191e57eff03691a5"
server_key_version: "encryptionkey@0"
I20260806 06:51:03.008862 9469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "64959a6e9fe847229b078aca10d1658d"
format_stamp: "Formatted at 2026-08-06 06:51:03 on dist-test-slave-0kls"
server_key: "ac59a1e1ef4b9cc2d023d7b421f6cb5e"
server_key_iv: "28692c1d15a312e6191e57eff03691a5"
server_key_version: "encryptionkey@0"
I20260806 06:51:03.010756 9469 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 06:51:03.011711 9483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:03.011936 9469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:51:03.012039 9469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "64959a6e9fe847229b078aca10d1658d"
format_stamp: "Formatted at 2026-08-06 06:51:03 on dist-test-slave-0kls"
server_key: "ac59a1e1ef4b9cc2d023d7b421f6cb5e"
server_key_iv: "28692c1d15a312e6191e57eff03691a5"
server_key_version: "encryptionkey@0"
I20260806 06:51:03.012117 9469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:03.035903 9469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:03.036253 9469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:03.036404 9469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:03.040334 9469 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:43907
I20260806 06:51:03.040366 9535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:43907 every 8 connection(s)
I20260806 06:51:03.040714 9469 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:03.041606 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:03.043875 9536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d: Bootstrap starting.
I20260806 06:51:03.044390 9536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:03.044610 9536 log.cc:824] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:03.045204 9536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d: No bootstrap required, opened a new log
I20260806 06:51:03.046769 9536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } }
I20260806 06:51:03.046904 9536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:03.046926 9536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64959a6e9fe847229b078aca10d1658d, State: Initialized, Role: FOLLOWER
I20260806 06:51:03.047062 9536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } }
I20260806 06:51:03.047137 9536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:03.047158 9536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:03.047185 9536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:03.047646 9536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } }
I20260806 06:51:03.047724 9536 leader_election.cc:302] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64959a6e9fe847229b078aca10d1658d; no voters:
I20260806 06:51:03.047845 9536 leader_election.cc:288] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:03.047904 9539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:03.048098 9539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [term 1 LEADER]: Becoming Leader. State: Replica: 64959a6e9fe847229b078aca10d1658d, State: Running, Role: LEADER
I20260806 06:51:03.048126 9536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:03.048197 9539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } }
I20260806 06:51:03.048581 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64959a6e9fe847229b078aca10d1658d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } } }
I20260806 06:51:03.048730 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:03.048714 9541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64959a6e9fe847229b078aca10d1658d. Latest consensus state: current_term: 1 leader_uuid: "64959a6e9fe847229b078aca10d1658d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64959a6e9fe847229b078aca10d1658d" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43907 } } }
I20260806 06:51:03.049211 9541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:03.049238 9548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:03.049748 9548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:03.050114 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9469
I20260806 06:51:03.050209 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:03.050410 4062 external_mini_cluster.cc:1428] Setting key 86738bcbc561b6e8fa09fd9e0bdce174
I20260806 06:51:03.051461 9548 catalog_manager.cc:1347] Generated new cluster ID: c410b609e0e74cc49988857cbeb8feb3
I20260806 06:51:03.051522 9548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:03.056522 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:03.050979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.3.247.190:43907)
0806 06:51:03.051178 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:03.051186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:03.051238 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:51:03.051414 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:03.051422 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:03.051493 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:03.051629 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:03.051943 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.053341 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.054426 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.055066 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.055189 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:03.055849 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:03.055853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:03.055853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:03.056054 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:03.056093 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:03.056107 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:03.056175 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:03.056237 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:51:03.056309 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260806 06:51:03.057443 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:51:03.081499 9548 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:03.082103 9548 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:03.086736 9548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64959a6e9fe847229b078aca10d1658d: Generated new TSK 0
I20260806 06:51:03.086928 9548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:03.180426 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:03.098126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.3.247.190:43907)
0806 06:51:03.098176 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:03.098178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:03.098244 (+ 66us) server_negotiation.cc:407] Connection header received
0806 06:51:03.098310 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:03.098311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:03.098323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:03.098399 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:51:03.098519 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.099299 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.100119 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.100432 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.100512 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:03.100657 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:51:03.180221 (+ 79564us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:51:03.180371 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58032: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:51:03.180541 9559 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58032: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 06:51:03.180557 9565 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:03.098055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43907 (local address 127.0.0.1:58032)
0806 06:51:03.098183 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:03.098195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:03.098255 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:03.098401 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:03.098402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:03.098403 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:51:03.098451 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:03.098453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.099392 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.099392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:03.100032 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:03.100035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:03.100515 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:03.100517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:03.100555 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:03.100560 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:51:03.180352 (+ 79792us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:51:03.180474 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.247.190:43907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260806 06:51:03.181058 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9469
2026-08-06T06:51:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (301 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:03 dist-test-slave-0kls krb5kdc[9577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:03 dist-test-slave-0kls krb5kdc[9577](info): set up 2 sockets
Aug 06 06:51:03 dist-test-slave-0kls krb5kdc[9577](info): commencing operation
krb5kdc: starting...
W20260806 06:51:04.406554 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:04 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:51:04.525554 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:46567
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:46567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:04.607762 9590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:04.608074 9590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:04.608114 9590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:04.610545 9590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:04.610610 9590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:04.610636 9590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:04.610661 9590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:04.610684 9590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:04.613682 9590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:46567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:46567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:04.614012 9590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:04.614274 9590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:04.619355 9595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:04.619382 9596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:04.619467 9598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:04.619773 9590 server_base.cc:1034] running on GCE node
I20260806 06:51:04.620214 9590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:04.620579 9590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:04.621757 9590 hybrid_clock.cc:648] HybridClock initialized: now 1785999064621747 us; error 31 us; skew 500 ppm
Aug 06 06:51:04 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999064, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:04.624284 9590 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:51:04.624800 9590 webserver.cc:460] Webserver started at http://127.3.247.190:43537/ using document root <none> and password file <none>
I20260806 06:51:04.625049 9590 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:04.625119 9590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:04.625267 9590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:04.626413 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "7606b8cffbd84f12b974160ddd485bc8"
format_stamp: "Formatted at 2026-08-06 06:51:04 on dist-test-slave-0kls"
server_key: "07d308fc5c10813c20807f78e271d2d0"
server_key_iv: "3e040b347babbc58f96b670e8a1570c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:04.626782 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "7606b8cffbd84f12b974160ddd485bc8"
format_stamp: "Formatted at 2026-08-06 06:51:04 on dist-test-slave-0kls"
server_key: "07d308fc5c10813c20807f78e271d2d0"
server_key_iv: "3e040b347babbc58f96b670e8a1570c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:04.628134 9590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:51:04.628881 9605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:04.629156 9590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:51:04.629263 9590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "7606b8cffbd84f12b974160ddd485bc8"
format_stamp: "Formatted at 2026-08-06 06:51:04 on dist-test-slave-0kls"
server_key: "07d308fc5c10813c20807f78e271d2d0"
server_key_iv: "3e040b347babbc58f96b670e8a1570c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:04.629338 9590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:04.635784 9590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:04.637848 9590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:04.638010 9590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:04.642278 9590 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:46567
I20260806 06:51:04.642313 9657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:46567 every 8 connection(s)
I20260806 06:51:04.642715 9590 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:04.643606 9658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:04.646063 9658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8: Bootstrap starting.
I20260806 06:51:04.646641 9658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:04.646900 9658 log.cc:824] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:04.647536 9658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8: No bootstrap required, opened a new log
I20260806 06:51:04.649159 9658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } }
I20260806 06:51:04.649317 9658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:04.649340 9658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7606b8cffbd84f12b974160ddd485bc8, State: Initialized, Role: FOLLOWER
I20260806 06:51:04.649452 9658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } }
I20260806 06:51:04.649505 9658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:04.649526 9658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:04.649607 9658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:04.650597 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9590
I20260806 06:51:04.650713 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:04.650895 4062 external_mini_cluster.cc:1428] Setting key 2df922d6763aab160aaa5552c85bf8fa
I20260806 06:51:04.652846 9658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } }
I20260806 06:51:04.652968 9658 leader_election.cc:302] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7606b8cffbd84f12b974160ddd485bc8; no voters:
I20260806 06:51:04.653149 9658 leader_election.cc:288] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:04.653299 9663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:04.653407 9658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:04.653510 9663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [term 1 LEADER]: Becoming Leader. State: Replica: 7606b8cffbd84f12b974160ddd485bc8, State: Running, Role: LEADER
I20260806 06:51:04.653649 9663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } }
I20260806 06:51:04.654397 9663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7606b8cffbd84f12b974160ddd485bc8. Latest consensus state: current_term: 1 leader_uuid: "7606b8cffbd84f12b974160ddd485bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } } }
I20260806 06:51:04.654481 9663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:04.655046 9672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:04.655405 9665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7606b8cffbd84f12b974160ddd485bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7606b8cffbd84f12b974160ddd485bc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 46567 } } }
I20260806 06:51:04.655478 9665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:04.655655 9672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:04.657959 9672 catalog_manager.cc:1347] Generated new cluster ID: 0e1bcf5da4254565b57bc32ac92aa491
I20260806 06:51:04.658030 9672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:51:04 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:04.663511 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:04.651381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35290 (local address 127.3.247.190:46567)
0806 06:51:04.651582 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:04.651602 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:04.651628 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:51:04.651724 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:04.651730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:04.651773 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:04.651896 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:04.652481 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:04.654492 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:04.655700 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:04.656459 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:04.656892 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:04.659524 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:04.659528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:04.659529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:04.659542 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:04.661734 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:04.662183 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:04.662187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:04.662187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:04.662321 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:04.662532 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:04.662533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:04.662533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:04.662881 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:04.663053 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:04.663126 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:51:04.663272 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260806 06:51:04.672515 9672 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:04.673134 9672 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:04.676498 9672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7606b8cffbd84f12b974160ddd485bc8: Generated new TSK 0
I20260806 06:51:04.676761 9672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
W20260806 06:51:05.975400 4062 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.247.129: real 1.296s user 0.000s sys 0.003s
I20260806 06:51:05.975751 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:06.093690 9684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:06.094071 9684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:06.094113 9684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:06.095469 9684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:06.095520 9684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:06.095549 9684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:06.097045 9684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:06.097240 9684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:06.097420 9684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:06.102650 9689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:06.102655 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:06.102665 9690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:06.102846 9684 server_base.cc:1034] running on GCE node
I20260806 06:51:06.103098 9684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:06.103350 9684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:06.104526 9684 hybrid_clock.cc:648] HybridClock initialized: now 1785999066104503 us; error 35 us; skew 500 ppm
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.107054 9684 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:51:06.107529 9684 webserver.cc:460] Webserver started at http://127.3.247.129:34189/ using document root <none> and password file <none>
I20260806 06:51:06.107761 9684 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:06.107831 9684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:06.107985 9684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:06.109189 9684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "2661e801d59245c8a393197f0a3d262d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "f2448fe1f61bc1c21a1d25a28e955f9e"
server_key_iv: "e18068043d5cf6f8ed976bdf052bd7d0"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.109571 9684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "2661e801d59245c8a393197f0a3d262d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "f2448fe1f61bc1c21a1d25a28e955f9e"
server_key_iv: "e18068043d5cf6f8ed976bdf052bd7d0"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.110971 9684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:06.111680 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.111843 9684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:06.111943 9684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "2661e801d59245c8a393197f0a3d262d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "f2448fe1f61bc1c21a1d25a28e955f9e"
server_key_iv: "e18068043d5cf6f8ed976bdf052bd7d0"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.112026 9684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:06.141970 9684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:06.144044 9684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:06.144225 9684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:06.144480 9684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:06.144807 9684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:06.144865 9684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.144907 9684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:06.144939 9684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.150540 9684 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:46601
I20260806 06:51:06.150580 9812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:46601 every 8 connection(s)
I20260806 06:51:06.150907 9684 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:06.152918 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9684
I20260806 06:51:06.153040 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:06.153251 4062 external_mini_cluster.cc:1428] Setting key d86ea5cbdc31ebe830370f88a4bf75b4
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:51:06.162286 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.151666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:33329 (local address 127.3.247.190:46567)
0806 06:51:06.151793 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.151795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.151974 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:51:06.153260 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.153261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.153276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.153341 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.154352 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.154953 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.156260 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.156493 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.156552 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.159175 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.159176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.159177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.159183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.160682 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.161166 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.161166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.161167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.161220 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.161508 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.161508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.161509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.161633 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.161689 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.162085 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.162159 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260806 06:51:06.162492 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.151635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:46567 (local address 127.3.247.129:33329)
0806 06:51:06.151852 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:06.151888 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:06.153036 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:06.153473 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:06.153478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:06.153913 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:06.154226 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.154233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.155038 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.155039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.156165 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.156168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.156584 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.156593 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.156742 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.157255 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:06.157266 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:06.159077 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:06.160768 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.160775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.160778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.161091 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.161286 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.161288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.161289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.161434 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.161718 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:06.161725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:06.161932 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 06:51:06.162052 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 06:51:06.162234 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260806 06:51:06.162997 9813 heartbeater.cc:347] Connected to a master server at 127.3.247.190:46567
I20260806 06:51:06.163151 9813 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:06.163339 9813 heartbeater.cc:511] Master 127.3.247.190:46567 requested a full tablet report, sending...
I20260806 06:51:06.163856 9622 ts_manager.cc:194] Registered new tserver with Master: 2661e801d59245c8a393197f0a3d262d (127.3.247.129:46601)
I20260806 06:51:06.164461 9622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:33329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
W20260806 06:51:06.462553 4062 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.247.130: real 0.308s user 0.000s sys 0.003s
I20260806 06:51:06.463028 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:06.543860 9819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:06.544129 9819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:06.544166 9819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:06.545370 9819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:06.545455 9819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:06.545495 9819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:06.546988 9819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:06.547255 9819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:06.547494 9819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:06.552518 9824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:06.552727 9819 server_base.cc:1034] running on GCE node
W20260806 06:51:06.552503 9827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:06.552486 9825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:06.553120 9819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:06.553402 9819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:06.554566 9819 hybrid_clock.cc:648] HybridClock initialized: now 1785999066554554 us; error 43 us; skew 500 ppm
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.557015 9819 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:51:06.557462 9819 webserver.cc:460] Webserver started at http://127.3.247.130:33505/ using document root <none> and password file <none>
I20260806 06:51:06.557685 9819 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:06.557729 9819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:06.557919 9819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:06.558964 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "66df8b7e97d2459e87331ad57686196d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "0ab94aa836a2e16c2ff5a0bc4aa7ba8c"
server_key_iv: "c5b5af931e4dee9460efa4e951e90d48"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.559325 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "66df8b7e97d2459e87331ad57686196d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "0ab94aa836a2e16c2ff5a0bc4aa7ba8c"
server_key_iv: "c5b5af931e4dee9460efa4e951e90d48"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.560602 9819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:06.561316 9834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.561487 9819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:06.561566 9819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "66df8b7e97d2459e87331ad57686196d"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "0ab94aa836a2e16c2ff5a0bc4aa7ba8c"
server_key_iv: "c5b5af931e4dee9460efa4e951e90d48"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.561621 9819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:06.587144 9819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:06.588992 9819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:06.589109 9819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:06.589325 9819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:06.589635 9819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:06.589669 9819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.589692 9819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:06.589740 9819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.595270 9819 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:46827
I20260806 06:51:06.595350 9947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:46827 every 8 connection(s)
I20260806 06:51:06.595656 9819 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:06.597919 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9819
I20260806 06:51:06.598026 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:06.598227 4062 external_mini_cluster.cc:1428] Setting key 209360821c88cb4605df8a96608d90a6
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:06.606151 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.596413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:35203 (local address 127.3.247.190:46567)
0806 06:51:06.596481 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.596483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.596671 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:51:06.597582 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.597583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.597599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.597653 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.598575 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.599216 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.600332 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.600525 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.600588 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.603004 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.603007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.603007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.603015 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.604444 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.604920 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.604921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.604921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.604965 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.605252 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.605253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.605253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.605350 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.605399 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.605758 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.606085 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:51:06.606189 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.596367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:46567 (local address 127.3.247.130:35203)
0806 06:51:06.596538 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:06.596580 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:06.597492 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:06.597754 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:06.597759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:06.598178 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:06.598465 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.598471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.599308 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.599309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.600244 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.600248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.600610 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.600611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.600762 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.601275 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:06.601285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:06.602887 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:06.604524 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.604531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.604534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.604843 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.605030 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.605031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.605031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.605184 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.605415 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:06.605422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:06.605597 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 06:51:06.605725 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:51:06.605916 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 06:51:06.606665 9948 heartbeater.cc:347] Connected to a master server at 127.3.247.190:46567
I20260806 06:51:06.606777 9948 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:06.606947 9948 heartbeater.cc:511] Master 127.3.247.190:46567 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
I20260806 06:51:06.607314 9622 ts_manager.cc:194] Registered new tserver with Master: 66df8b7e97d2459e87331ad57686196d (127.3.247.130:46827)
I20260806 06:51:06.607710 9622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:35203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:51:06.624434 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:06.705283 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:06.705472 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:06.705495 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:06.706717 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:06.706759 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:06.706787 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:06.708272 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:46567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:06.708511 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:06.708751 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:06.713603 9958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:06.713639 9961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:06.713636 9959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:06.713819 9953 server_base.cc:1034] running on GCE node
I20260806 06:51:06.714084 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:06.714316 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:06.715498 9953 hybrid_clock.cc:648] HybridClock initialized: now 1785999066715478 us; error 34 us; skew 500 ppm
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.718111 9953 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:51:06.718552 9953 webserver.cc:460] Webserver started at http://127.3.247.131:39679/ using document root <none> and password file <none>
I20260806 06:51:06.718753 9953 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:06.718794 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:06.718884 9953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:06.719905 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "0123fcf844214907a3aee36a4644192e"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "5ee8fabf41d62019cfc52c19cac22fd4"
server_key_iv: "dca23d78f88f063a4f0d3c2c020be9d1"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.720208 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "0123fcf844214907a3aee36a4644192e"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "5ee8fabf41d62019cfc52c19cac22fd4"
server_key_iv: "dca23d78f88f063a4f0d3c2c020be9d1"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.721397 9953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:51:06.722128 9968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.722280 9953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:51:06.722360 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "0123fcf844214907a3aee36a4644192e"
format_stamp: "Formatted at 2026-08-06 06:51:06 on dist-test-slave-0kls"
server_key: "5ee8fabf41d62019cfc52c19cac22fd4"
server_key_iv: "dca23d78f88f063a4f0d3c2c020be9d1"
server_key_version: "encryptionkey@0"
I20260806 06:51:06.722414 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:06.734478 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:06.736833 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:06.736979 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:06.737237 9953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:06.737557 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:06.737592 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.737615 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:06.737665 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:06.743335 9953 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:36147
I20260806 06:51:06.743371 10081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:36147 every 8 connection(s)
I20260806 06:51:06.743726 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:06.749179 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 9953
I20260806 06:51:06.749289 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:06.749495 4062 external_mini_cluster.cc:1428] Setting key 74c2d0956bfc0a33e5ef0633e0e805fe
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:06.752799 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.744501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:34993 (local address 127.3.247.190:46567)
0806 06:51:06.744571 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.744589 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.744765 (+ 176us) server_negotiation.cc:407] Connection header received
0806 06:51:06.745706 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.745707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.745719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.745778 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.746389 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.746901 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.747877 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.748091 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.748176 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.750525 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.750526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.750526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.750532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.751617 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.751979 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.751979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.751980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.752031 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.752238 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.752239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.752239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.752337 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.752407 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.752622 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.752692 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:51:06.753047 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.744481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:46567 (local address 127.3.247.131:34993)
0806 06:51:06.744648 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:06.744682 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:06.745592 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:06.745865 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:06.745868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:06.746114 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:06.746293 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.746297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.746994 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.746995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.747788 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:06.747790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.748186 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.748187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:06.748302 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.748723 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:06.748730 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:06.750441 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:06.751684 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.751689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.751690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.751886 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.752087 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:06.752087 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:06.752088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:06.752186 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:06.752415 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:06.752418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:06.752522 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 06:51:06.752609 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:51:06.752790 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260806 06:51:06.753484 10082 heartbeater.cc:347] Connected to a master server at 127.3.247.190:46567
I20260806 06:51:06.753577 10082 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:06.753824 10082 heartbeater.cc:511] Master 127.3.247.190:46567 requested a full tablet report, sending...
I20260806 06:51:06.754240 9622 ts_manager.cc:194] Registered new tserver with Master: 0123fcf844214907a3aee36a4644192e (127.3.247.131:36147)
I20260806 06:51:06.754616 9622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:34993
I20260806 06:51:06.761382 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/test-admin.keytab.
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.777554 4062 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 06:51:06.779245 10089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:06.778487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.3.247.129:46601)
0806 06:51:06.778626 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.778630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.778642 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:51:06.778687 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.778689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.778703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.778767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.779157 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42244: BlockingRecv error: recv got EOF from 127.0.0.1:42244 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
W20260806 06:51:06.779448 4062 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc)
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.780431 4062 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.781301 4062 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:51:06.788699 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.781858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.3.247.129:46601)
0806 06:51:06.781920 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.781922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.781967 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:51:06.782019 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.782020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.782031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.782085 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.782461 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.783164 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.783828 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.784146 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.784203 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.786313 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.786320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.786320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.786326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.787669 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.787970 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.787972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.787973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.788024 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.788194 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.788195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.788195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.788315 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.788437 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.788470 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.788527 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:51:06.790555 10089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:06.789938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.3.247.129:46601)
0806 06:51:06.790004 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.790006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.790064 (+ 58us) server_negotiation.cc:407] Connection header received
0806 06:51:06.790143 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.790144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.790154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.790219 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.790485 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42258: BlockingRecv error: recv got EOF from 127.0.0.1:42258 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 06:51:06.790678 4062 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.791641 4062 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.792445 4062 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:51:06.799220 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.792964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.3.247.129:46601)
0806 06:51:06.793020 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.793021 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.793064 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:51:06.793109 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.793110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.793119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.793164 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.793487 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.794009 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.794683 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.794896 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.794970 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.796913 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.796914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.796914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.796921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.798229 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.798531 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.798532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.798532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.798588 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.798757 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.798758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.798758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.798847 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.798945 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.799003 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.799064 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 06:51:06.800709 10089 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:06.800192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.3.247.129:46601)
0806 06:51:06.800247 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.800248 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.800284 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:51:06.800355 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.800355 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.800363 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.800410 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.800647 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42274: BlockingRecv error: recv got EOF from 127.0.0.1:42274 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 06:51:06.800824 4062 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785999040366271-4062-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.801923 4062 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:06.802779 4062 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[9577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:51:06.809489 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:06.803256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.3.247.129:46601)
0806 06:51:06.803314 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:06.803316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:06.803324 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:06.803367 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:06.803368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:06.803375 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:06.803433 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:06.803775 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.804242 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.804911 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:06.805113 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:06.805172 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:06.807116 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:06.807117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:06.807117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:06.807123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:06.808520 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.808869 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.808870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.808870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.808945 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:06.809125 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:06.809126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:06.809126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:06.809215 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:06.809318 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:06.809369 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:51:06.809429 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:51:06.810129 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9684
I20260806 06:51:06.815774 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9819
I20260806 06:51:06.819949 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9953
I20260806 06:51:06.824120 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 9590
2026-08-06T06:51:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3460 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[10102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[10102](info): set up 2 sockets
Aug 06 06:51:06 dist-test-slave-0kls krb5kdc[10102](info): commencing operation
krb5kdc: starting...
W20260806 06:51:07.854022 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:07 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab.
Entry for principal oryx/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab.
I20260806 06:51:07.968503 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36303
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab
--principal=oryx/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:08.052366 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:08.052609 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:08.052664 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:08.053917 10115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:08.053977 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:08.053999 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:08.054020 10115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:08.054049 10115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:08.055537 10115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab
--principal=oryx/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:08.055820 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:08.056087 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.060896 10123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.060938 10121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.060901 10120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.061172 10115 server_base.cc:1034] running on GCE node
I20260806 06:51:08.061430 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.061704 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.062861 10115 hybrid_clock.cc:648] HybridClock initialized: now 1785999068062861 us; error 45 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.069866 10115 init.cc:377] Logged in from keytab as oryx/127.3.247.190@KRBTEST.COM (short username oryx)
I20260806 06:51:08.070287 10115 webserver.cc:460] Webserver started at http://127.3.247.190:45565/ using document root <none> and password file <none>
I20260806 06:51:08.070521 10115 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.070586 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.070750 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:08.071748 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "675f3ce109c2436bb34fa69d23985b8c"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e6a8320554ccbe50c7215f83c5ceb9f6"
server_key_iv: "f319b5fb1cadff726f0e94cdd12a75c0"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.072085 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "675f3ce109c2436bb34fa69d23985b8c"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e6a8320554ccbe50c7215f83c5ceb9f6"
server_key_iv: "f319b5fb1cadff726f0e94cdd12a75c0"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.073405 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:08.074155 10130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.074312 10115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.074416 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "675f3ce109c2436bb34fa69d23985b8c"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e6a8320554ccbe50c7215f83c5ceb9f6"
server_key_iv: "f319b5fb1cadff726f0e94cdd12a75c0"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.074488 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.083680 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.085924 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:08.086086 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.089692 10115 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:36303
I20260806 06:51:08.089725 10182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:36303 every 8 connection(s)
I20260806 06:51:08.090127 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:08.091051 10183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:08.092681 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10115
I20260806 06:51:08.092808 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:08.092998 4062 external_mini_cluster.cc:1428] Setting key cc82182f7ee6947aed0b75a9efe493dc
I20260806 06:51:08.093614 10183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Bootstrap starting.
I20260806 06:51:08.094238 10183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:08.094607 10183 log.cc:824] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:08.095347 10183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: No bootstrap required, opened a new log
I20260806 06:51:08.097458 10183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } }
I20260806 06:51:08.097656 10183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:08.097707 10183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f3ce109c2436bb34fa69d23985b8c, State: Initialized, Role: FOLLOWER
I20260806 06:51:08.097882 10183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } }
I20260806 06:51:08.097990 10183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:08.098027 10183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:08.098083 10183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:08.098822 10183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } }
I20260806 06:51:08.098939 10183 leader_election.cc:302] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 675f3ce109c2436bb34fa69d23985b8c; no voters:
I20260806 06:51:08.099120 10183 leader_election.cc:288] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:08.099174 10188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:08.099313 10188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 LEADER]: Becoming Leader. State: Replica: 675f3ce109c2436bb34fa69d23985b8c, State: Running, Role: LEADER
I20260806 06:51:08.099376 10183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:08.099418 10188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } }
I20260806 06:51:08.099737 10188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675f3ce109c2436bb34fa69d23985b8c. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } }
I20260806 06:51:08.099828 10188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:08.099747 10190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } }
I20260806 06:51:08.099964 10190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:08.100023 10194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:08.100494 10194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:08.103497 10194 catalog_manager.cc:1347] Generated new cluster ID: 0f0abea884df4bd193d6b990f9fa8c5c
I20260806 06:51:08.103554 10194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:08.104396 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.093528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.3.247.190:36303)
0806 06:51:08.093694 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.093704 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.093737 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:51:08.094217 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.094223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.094268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.094420 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.095200 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.096102 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.097215 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.097608 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.097717 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.100684 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.100687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.100688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.100700 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.102769 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.103195 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.103197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.103197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.103275 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.103487 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.103488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.103489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.103759 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.103914 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.104025 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.104160 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260806 06:51:08.127183 10194 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:08.127947 10194 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:08.139415 10194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Generated new TSK 0
I20260806 06:51:08.139622 10194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
I20260806 06:51:08.173651 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:08.284781 10209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:08.284982 10209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:08.285005 10209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:08.286258 10209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:08.286299 10209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:08.286351 10209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:08.287817 10209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:08.288059 10209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:08.288298 10209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.293159 10215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.293167 10214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.293165 10217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.293377 10209 server_base.cc:1034] running on GCE node
I20260806 06:51:08.293581 10209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.293929 10209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.295071 10209 hybrid_clock.cc:648] HybridClock initialized: now 1785999068295062 us; error 27 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.297536 10209 init.cc:377] Logged in from keytab as oryx/127.3.247.129@KRBTEST.COM (short username oryx)
I20260806 06:51:08.298002 10209 webserver.cc:460] Webserver started at http://127.3.247.129:44691/ using document root <none> and password file <none>
I20260806 06:51:08.298215 10209 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.298257 10209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.298344 10209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:08.299480 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "49496af4abc44429a3874ca583ee3b62"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "16654c7c27d0b04a0c154b61d44cf7c0"
server_key_iv: "80a5034bf0e206f76279f70758be532c"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.299785 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "49496af4abc44429a3874ca583ee3b62"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "16654c7c27d0b04a0c154b61d44cf7c0"
server_key_iv: "80a5034bf0e206f76279f70758be532c"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.301118 10209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:51:08.301821 10224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.302021 10209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:08.302106 10209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "49496af4abc44429a3874ca583ee3b62"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "16654c7c27d0b04a0c154b61d44cf7c0"
server_key_iv: "80a5034bf0e206f76279f70758be532c"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.302160 10209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.333913 10209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.335709 10209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:08.335820 10209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.336006 10209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:08.336346 10209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:08.336380 10209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.336401 10209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:08.336453 10209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.341974 10209 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:37119
I20260806 06:51:08.342056 10337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:37119 every 8 connection(s)
I20260806 06:51:08.342386 10209 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:08.349609 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10209
I20260806 06:51:08.349723 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:08.349998 4062 external_mini_cluster.cc:1428] Setting key 3c4f66560dfa9a60263f614bfe66ddea
I20260806 06:51:08.352128 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.343184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:56281 (local address 127.3.247.190:36303)
0806 06:51:08.343251 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.343253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.343477 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:51:08.343951 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.343952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.343961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.344025 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.344941 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.345440 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.346407 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.346611 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.346663 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.349066 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.349068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.349069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.349077 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.350724 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.351122 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.351124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.351124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.351183 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.351445 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.351446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.351446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.351560 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.351653 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.351985 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.352058 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:51:08.352667 10340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.343156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:56281)
0806 06:51:08.343336 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.343398 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.343871 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.344085 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.344088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.344677 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.344865 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.344869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.345520 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.345520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.346337 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.346338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.346683 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.346684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.346783 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.347232 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.347253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.348951 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.350809 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.350817 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.350819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.351042 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.351257 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.351257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.351258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.351374 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.351644 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.351651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.351827 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.351945 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.352121 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260806 06:51:08.353183 10338 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:08.353264 10338 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:08.353466 10338 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:08.354034 10147 ts_manager.cc:194] Registered new tserver with Master: 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119)
I20260806 06:51:08.354682 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:56281
WARNING: no policy specified for oryx/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
I20260806 06:51:08.376099 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:08.457602 10343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:08.457870 10343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:08.457911 10343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:08.459077 10343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:08.459162 10343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:08.459208 10343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:08.460662 10343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:08.460937 10343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:08.461179 10343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.466104 10349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.466120 10348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.466116 10351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.466444 10343 server_base.cc:1034] running on GCE node
I20260806 06:51:08.466662 10343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.466866 10343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.468017 10343 hybrid_clock.cc:648] HybridClock initialized: now 1785999068468007 us; error 34 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.470736 10343 init.cc:377] Logged in from keytab as oryx/127.3.247.130@KRBTEST.COM (short username oryx)
I20260806 06:51:08.471189 10343 webserver.cc:460] Webserver started at http://127.3.247.130:38381/ using document root <none> and password file <none>
I20260806 06:51:08.471412 10343 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.471457 10343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.471627 10343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:08.472718 10343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "495bed5977624d109524bc9811388206"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "f2113de092ab0f293a81576045a82ecf"
server_key_iv: "7dc482bbdad066d03f9435d76b805373"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.473071 10343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "495bed5977624d109524bc9811388206"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "f2113de092ab0f293a81576045a82ecf"
server_key_iv: "7dc482bbdad066d03f9435d76b805373"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.474475 10343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:08.475150 10358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.475309 10343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:08.475411 10343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "495bed5977624d109524bc9811388206"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "f2113de092ab0f293a81576045a82ecf"
server_key_iv: "7dc482bbdad066d03f9435d76b805373"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.475486 10343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.485299 10343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.487671 10343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:08.487835 10343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.488071 10343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:08.488390 10343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:08.488442 10343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.488482 10343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:08.488510 10343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.494292 10343 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:36913
I20260806 06:51:08.494349 10471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:36913 every 8 connection(s)
I20260806 06:51:08.494695 10343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:08.500746 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10343
I20260806 06:51:08.500882 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:08.501098 4062 external_mini_cluster.cc:1428] Setting key d83b17cab881250310ab7d4a6f8204e5
I20260806 06:51:08.504392 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.495442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:60157 (local address 127.3.247.190:36303)
0806 06:51:08.495489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.495492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.495679 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:51:08.496207 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.496208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.496220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.496279 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.497230 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.497736 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.498785 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.498984 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.499070 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.501321 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.501322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.501323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.501329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.503006 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.503435 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.503436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.503437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.503498 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.503762 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.503763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.503763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.503873 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.503949 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.504219 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.504298 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:08.504657 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.495403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:60157)
0806 06:51:08.495538 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.495585 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.496102 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.496347 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.496351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.496924 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.497124 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.497130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.497863 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.497864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.498660 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.498662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.499085 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.499086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.499207 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.499651 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.499658 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.501199 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.503087 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.503093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.503094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.503355 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.503566 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.503567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.503568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.503684 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.503941 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.503946 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.504086 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.504198 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.504357 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260806 06:51:08.505116 10472 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:08.505225 10472 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:08.505398 10472 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:08.505843 10147 ts_manager.cc:194] Registered new tserver with Master: 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:08.506378 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:60157
WARNING: no policy specified for oryx/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
I20260806 06:51:08.528894 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:08.608510 10477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:08.608721 10477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:08.608744 10477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:08.609975 10477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:08.610018 10477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:08.610066 10477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:08.611531 10477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:08.611778 10477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:08.612011 10477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.617013 10485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.617044 10483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.617183 10477 server_base.cc:1034] running on GCE node
W20260806 06:51:08.617044 10482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.617497 10477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.617753 10477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.618945 10477 hybrid_clock.cc:648] HybridClock initialized: now 1785999068618937 us; error 31 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.621421 10477 init.cc:377] Logged in from keytab as oryx/127.3.247.131@KRBTEST.COM (short username oryx)
I20260806 06:51:08.621950 10477 webserver.cc:460] Webserver started at http://127.3.247.131:45371/ using document root <none> and password file <none>
I20260806 06:51:08.622197 10477 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.622264 10477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.622414 10477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:08.623461 10477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "6ca86b739b714c618525f500c7c91ac1"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e9b8b1af81819af30d7d2a3bf2a7d85e"
server_key_iv: "145415ae7b6be40e3099f80c7a824927"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.623816 10477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "6ca86b739b714c618525f500c7c91ac1"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e9b8b1af81819af30d7d2a3bf2a7d85e"
server_key_iv: "145415ae7b6be40e3099f80c7a824927"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.625087 10477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:51:08.625839 10492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.626017 10477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:08.626120 10477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "6ca86b739b714c618525f500c7c91ac1"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e9b8b1af81819af30d7d2a3bf2a7d85e"
server_key_iv: "145415ae7b6be40e3099f80c7a824927"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.626206 10477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.649286 10477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.651787 10477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:08.651955 10477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.652210 10477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:08.652534 10477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:08.652587 10477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.652621 10477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:08.652662 10477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.658605 10477 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:45657
I20260806 06:51:08.658660 10605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:45657 every 8 connection(s)
I20260806 06:51:08.658972 10477 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:08.663753 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10477
I20260806 06:51:08.663865 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:08.664054 4062 external_mini_cluster.cc:1428] Setting key c3929b85ababb0d927570011d88df274
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:08.667894 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.659696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:60871 (local address 127.3.247.190:36303)
0806 06:51:08.659763 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.659765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.659940 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:51:08.660426 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.660427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.660437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.660488 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.661405 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.661970 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.662922 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.663110 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.663164 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.665559 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.665560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.665560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.665566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.666786 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.667133 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.667134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.667134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.667184 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.667388 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.667388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.667389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.667487 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.667537 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.667768 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.667829 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:51:08.668114 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.659678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:60871)
0806 06:51:08.659834 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.659870 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.660343 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.660542 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.660545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.661124 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.661324 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.661329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.662047 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.662048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.662847 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.662849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.663182 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.663183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.663277 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.663835 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.663844 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.665486 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.666855 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.666860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.666862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.667070 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.667240 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.667241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.667241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.667336 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.667540 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.667543 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.667657 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.667740 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.667854 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260806 06:51:08.668666 10606 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:08.668768 10606 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:08.668915 10606 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:08.669281 10147 ts_manager.cc:194] Registered new tserver with Master: 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:08.669646 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:60871
I20260806 06:51:08.675971 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.247.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.189.keytab.
Entry for principal oryx/127.3.247.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.189.keytab.
I20260806 06:51:08.701149 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:37323
--webserver_interface=127.3.247.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.189.keytab
--principal=oryx/127.3.247.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36303,127.3.247.189:37323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:08.780829 10611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:08.781045 10611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:08.781068 10611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:08.782245 10611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:08.782285 10611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:08.782299 10611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:08.782353 10611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:08.782379 10611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:08.783859 10611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36303,127.3.247.189:37323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.189.keytab
--principal=oryx/127.3.247.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:37323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.247.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
W20260806 06:51:08.783987 10611 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.247.190:36303,127.3.247.189:37323'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 06:51:08.784148 10611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:08.784386 10611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.789040 10617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.789068 10616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.789260 10611 server_base.cc:1034] running on GCE node
W20260806 06:51:08.789041 10619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.789563 10611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.789875 10611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.791039 10611 hybrid_clock.cc:648] HybridClock initialized: now 1785999068791016 us; error 37 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.793421 10611 init.cc:377] Logged in from keytab as oryx/127.3.247.189@KRBTEST.COM (short username oryx)
I20260806 06:51:08.793965 10611 webserver.cc:460] Webserver started at http://127.3.247.189:33589/ using document root <none> and password file <none>
I20260806 06:51:08.794222 10611 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.794293 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.794450 10611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:08.795527 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/instance:
uuid: "2b09e21849da4a27aeb9e02d6080daf3"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "60165bcdbd941d6cea2ba6b92728b807"
server_key_iv: "3ebff220456854115b7d3d943ecf530e"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.795894 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal/instance:
uuid: "2b09e21849da4a27aeb9e02d6080daf3"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "60165bcdbd941d6cea2ba6b92728b807"
server_key_iv: "3ebff220456854115b7d3d943ecf530e"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.797200 10611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:51:08.798005 10626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.798199 10611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:08.798283 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "2b09e21849da4a27aeb9e02d6080daf3"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "60165bcdbd941d6cea2ba6b92728b807"
server_key_iv: "3ebff220456854115b7d3d943ecf530e"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.798352 10611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.835079 10611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.836930 10611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:08.837091 10611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.841149 10611 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.189:37323
I20260806 06:51:08.841220 10678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.189:37323 every 8 connection(s)
I20260806 06:51:08.841584 10611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
I20260806 06:51:08.842563 10679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:08.843987 10679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:51:08.846557 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10611
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.189@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:08.853291 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.844622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.3.247.190:36303)
0806 06:51:08.844665 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.844667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.844877 (+ 210us) server_negotiation.cc:407] Connection header received
0806 06:51:08.845324 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.845325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.845333 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.845387 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.846052 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.846538 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.847885 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.848074 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.848156 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.850399 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.850400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.850401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.850407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.851840 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.852228 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.852229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.852230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.852290 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.852566 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.852567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.852568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.852732 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.852820 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.853123 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.853200 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:08.853477 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.844587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42368)
0806 06:51:08.844752 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.844794 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.845248 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.845461 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.845464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.845702 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.845962 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.845967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.846648 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.846649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.847799 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.847801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.848160 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.848161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.848288 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.848750 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.848758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.850316 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.851922 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.851929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.851931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.852147 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.852359 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.852360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.852360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.852464 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.852811 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.852818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.852965 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.853090 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.853241 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260806 06:51:08.854079 10679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:51:08.854799 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.847012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34938 (local address 127.3.247.189:37323)
0806 06:51:08.847200 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.847205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.847236 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:51:08.847277 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.847279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.847298 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.847370 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.847918 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.848577 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.849312 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.849570 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.849626 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.851954 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.851960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.851960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.851967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.853555 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.853995 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.853998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.853998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.854059 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.854264 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.854265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.854266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.854440 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.854580 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.854652 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.854707 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":134,"threads_started":1}
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.189@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:08.860541 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.854333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34944)
0806 06:51:08.854384 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.854394 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.854430 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.854911 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.854913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.855039 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.855087 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.855088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.855769 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.855770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.856211 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.856213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.856512 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.856513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.856557 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.856950 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.856955 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.857989 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.859800 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.859801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.859801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.859947 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.860129 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.860129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.860129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.860175 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.860339 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.860339 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.860368 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.860416 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.860481 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:51:08.860649 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.854386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.3.247.189:37323)
0806 06:51:08.854819 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.854821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.854826 (+ 5us) server_negotiation.cc:407] Connection header received
0806 06:51:08.854841 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.854841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.854848 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.854899 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.855156 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.855709 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.856275 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.856457 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.856502 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.858067 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.858068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.858069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.858074 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.859735 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.860024 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.860025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.860025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.860073 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.860212 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.860212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.860212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.860292 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.860336 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.860429 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.860522 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":405,"thread_start_us":97,"threads_started":1}
I20260806 06:51:08.861991 10679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Bootstrap starting.
I20260806 06:51:08.862582 10679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:08.862869 10679 log.cc:824] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:08.863530 10679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: No bootstrap required, opened a new log
I20260806 06:51:08.864886 10679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } }
I20260806 06:51:08.865002 10679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:08.865072 10679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b09e21849da4a27aeb9e02d6080daf3, State: Initialized, Role: FOLLOWER
I20260806 06:51:08.865190 10679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } }
I20260806 06:51:08.865476 10679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:08.865438 10686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } } }
I20260806 06:51:08.865670 10686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:51:08.867024 10697 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:51:08.867079 10697 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:51:08.868600 10611 master_runner.cc:428] Detected that this master 2b09e21849da4a27aeb9e02d6080daf3 is joining an existing cluster
I20260806 06:51:08.868666 10611 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.247.189:37323
I20260806 06:51:08.876493 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.869725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42376)
0806 06:51:08.870021 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.870029 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.870092 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.870223 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.870223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.870442 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.870503 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.870505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.871371 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.871372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.872015 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.872017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.872456 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.872529 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.872995 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.872999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.873458 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.875302 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.875303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.875593 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.875798 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.875799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.875857 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.876133 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.876134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.876176 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.876271 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.876376 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":78,"threads_started":1}
I20260806 06:51:08.876505 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.869767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.3.247.190:36303)
0806 06:51:08.869824 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.869826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.870070 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:51:08.870155 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.870156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.870168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.870221 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.870579 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.871291 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872098 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.872359 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872422 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.873555 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.873556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.873556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.873565 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.875192 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875667 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.875668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.875669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.875730 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875938 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.875939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.875939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.876055 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.876128 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.876312 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.876413 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:51:08.876752 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.869572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.3.247.189:37323)
0806 06:51:08.869605 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.869606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.869926 (+ 320us) server_negotiation.cc:407] Connection header received
0806 06:51:08.869982 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.869983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.869991 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.870068 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.870474 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.871256 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872291 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.872578 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872663 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.873595 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.873596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.873597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.873602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.875634 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875986 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.875987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.875987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.876031 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.876212 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.876213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.876214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.876331 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.876389 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.876565 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.876651 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:51:08.876942 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.869507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34948)
0806 06:51:08.869856 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.869867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.869926 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.870071 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.870072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.870329 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.870403 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.870405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.871322 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.871323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.872210 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.872212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.872673 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.872674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.872733 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.873155 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.873159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.873473 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.875717 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.875718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.875718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.875913 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.876092 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.876092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.876093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.876162 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.876410 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.876411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.876454 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.876784 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.876861 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":30,"thread_start_us":81,"threads_started":1}
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.190@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:08.893589 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.879512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34962)
0806 06:51:08.885917 (+ 6405us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.885971 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.886050 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.886199 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.886202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.886550 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.886610 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.886612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.887224 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.887225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.888180 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.888182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.888493 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.888494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.888579 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.889016 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.889021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.890881 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.892573 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.892579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.892581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.892798 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.892997 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.892997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.892998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.893052 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.893300 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.893306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.893350 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.893415 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.893504 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6370,"thread_start_us":72,"threads_started":1}
I20260806 06:51:08.893638 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.879568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.3.247.189:37323)
0806 06:51:08.879600 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.879602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.886046 (+ 6444us) server_negotiation.cc:407] Connection header received
0806 06:51:08.886125 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.886126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.886135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.886184 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.886688 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.887151 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.888259 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.888428 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.888474 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.890971 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.890973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.890973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.890981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.892488 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.892869 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.892870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.892871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.892931 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.893109 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.893109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.893110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.893239 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.893306 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.893462 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.893542 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:51:08.894332 10147 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.247.189:37323: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3"
member_type: NON_VOTER
last_known_addr {
host: "127.3.247.189"
port: 37323
}
attrs {
promote: true
}
}
dest_uuid: "675f3ce109c2436bb34fa69d23985b8c"
cas_config_opid_index: -1
I20260806 06:51:08.894661 10147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } }
I20260806 06:51:08.895135 10707 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } }
I20260806 06:51:08.895522 10653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:08.896059 10707 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.247.189:37323
I20260806 06:51:08.896160 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } }
I20260806 06:51:08.896235 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:08.896312 10611 master.cc:537] Master@127.3.247.189:37323 shutting down...
I20260806 06:51:08.896821 10653 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Refusing update from remote peer 675f3ce109c2436bb34fa69d23985b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 06:51:08.897290 10706 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:51:08.898131 10131 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 06:51:08.898511 10131 consensus_peers.cc:487] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c -> Peer 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189:37323): Couldn't send request to peer 2b09e21849da4a27aeb9e02d6080daf3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:51:08.899111 10611 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:08.899220 10611 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:08.899268 10611 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: stopping tablet replica
I20260806 06:51:08.912042 10611 master.cc:552] Master@127.3.247.189:37323 shutdown complete.
I20260806 06:51:08.912120 10611 master_runner.cc:305] Clearing existing system tablet
I20260806 06:51:08.912242 10611 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.912302 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.912955 10611 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:08.913439 10711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.913575 10611 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.913632 10611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.913673 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "2b09e21849da4a27aeb9e02d6080daf3"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "60165bcdbd941d6cea2ba6b92728b807"
server_key_iv: "3ebff220456854115b7d3d943ecf530e"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.914471 10611 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:08.915876 10611 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:51:08.915957 10611 log.cc:1196] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 06:51:08.916158 10611 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Deleting consensus metadata
I20260806 06:51:08.916297 10611 master_runner.cc:315] Copying system tablet from 127.3.247.190:36303
I20260806 06:51:08.916870 10611 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: tablet copy: Beginning tablet copy session from remote peer at address 127.3.247.190:36303
I20260806 06:51:08.923135 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.917215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42386)
0806 06:51:08.917312 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:08.917322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:08.917386 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:08.917535 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:08.917536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:08.917680 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:08.917736 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.917737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.918710 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.918711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.919253 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:08.919255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.919722 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.919722 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:08.919767 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.920243 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:08.920247 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:08.920523 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:08.922205 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.922206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.922206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.922364 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.922584 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:08.922585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:08.922585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:08.922637 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:08.922894 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:08.922894 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:08.922927 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 06:51:08.922975 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:51:08.923048 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":61,"threads_started":1}
I20260806 06:51:08.923151 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:08.917266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.3.247.190:36303)
0806 06:51:08.917314 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:08.917316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:08.917379 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:51:08.917451 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:08.917452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:08.917471 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:08.917532 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:08.917816 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.918632 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.919333 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:08.919648 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:08.919720 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:08.920606 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:08.920607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:08.920608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:08.920615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:08.922134 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.922431 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.922432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.922432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.922497 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:08.922695 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:08.922697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:08.922697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:08.922835 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:08.922899 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:08.922998 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:51:08.923067 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:51:08.923745 10167 tablet_copy_service.cc:140] P 675f3ce109c2436bb34fa69d23985b8c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2b09e21849da4a27aeb9e02d6080daf3 ({username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:42386)
I20260806 06:51:08.923825 10167 tablet_copy_service.cc:161] P 675f3ce109c2436bb34fa69d23985b8c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2b09e21849da4a27aeb9e02d6080daf3 at {username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:42386: session id = 2b09e21849da4a27aeb9e02d6080daf3-00000000000000000000000000000000
I20260806 06:51:08.924464 10167 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:51:08.925032 10611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:51:08.925949 10611 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: tablet copy: Starting download of 0 data blocks...
I20260806 06:51:08.926107 10611 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: tablet copy: Starting download of 1 WAL segments...
I20260806 06:51:08.926891 10611 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:51:08.927610 10167 tablet_copy_service.cc:342] P 675f3ce109c2436bb34fa69d23985b8c: Request end of tablet copy session 2b09e21849da4a27aeb9e02d6080daf3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:42386
I20260806 06:51:08.927665 10167 tablet_copy_service.cc:433] P 675f3ce109c2436bb34fa69d23985b8c: ending tablet copy session 2b09e21849da4a27aeb9e02d6080daf3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2b09e21849da4a27aeb9e02d6080daf3
I20260806 06:51:08.929931 10611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:08.931036 10719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:08.931037 10718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.931449 10611 server_base.cc:1034] running on GCE node
W20260806 06:51:08.937925 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:08.938082 10611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:08.938239 10611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:08.939359 10611 hybrid_clock.cc:648] HybridClock initialized: now 1785999068939346 us; error 32 us; skew 500 ppm
Aug 06 06:51:08 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:08.940587 10611 init.cc:377] Logged in from keytab as oryx/127.3.247.189@KRBTEST.COM (short username oryx)
I20260806 06:51:08.940856 10611 webserver.cc:460] Webserver started at http://127.3.247.189:46037/ using document root <none> and password file <none>
I20260806 06:51:08.940982 10611 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:08.941022 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:08.941610 10611 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:08.942138 10728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.942252 10611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:08.942296 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "2b09e21849da4a27aeb9e02d6080daf3"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "60165bcdbd941d6cea2ba6b92728b807"
server_key_iv: "3ebff220456854115b7d3d943ecf530e"
server_key_version: "encryptionkey@0"
I20260806 06:51:08.942353 10611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:08.955489 10611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:08.957180 10611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:08.960119 10611 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.189:37323
I20260806 06:51:08.960130 10780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.189:37323 every 8 connection(s)
I20260806 06:51:08.960431 10611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
I20260806 06:51:08.961323 10781 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:51:08.961882 10781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Bootstrap starting.
I20260806 06:51:08.964686 10781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:51:08.964896 10781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Bootstrap complete.
I20260806 06:51:08.965066 10781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } }
I20260806 06:51:08.965173 10781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b09e21849da4a27aeb9e02d6080daf3, State: Initialized, Role: LEARNER
I20260806 06:51:08.965212 10781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } }
I20260806 06:51:08.965394 10781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:08.965373 10784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } }
I20260806 06:51:08.965497 10784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:51:08.966984 10795 catalog_manager.cc:1259] Loaded cluster ID: 0f0abea884df4bd193d6b990f9fa8c5c
I20260806 06:51:08.967031 10795 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: loading cluster ID for follower catalog manager: success
I20260806 06:51:08.968081 10795 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: acquiring CA information for follower catalog manager: success
I20260806 06:51:08.968364 10795 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:51:09.013504 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.007429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.3.247.189:37323)
0806 06:51:09.007628 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.007630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.007637 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:09.007658 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.007659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.007668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.007751 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.008309 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.008960 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.009690 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.009945 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.010023 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.011154 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.011155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.011155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.011163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.012343 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.012717 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.012719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.012719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.012780 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.012976 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.012977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.012978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.013093 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.013194 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.013311 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.013390 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for oryx/127.3.247.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.188.keytab.
Entry for principal oryx/127.3.247.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.188.keytab.
I20260806 06:51:09.037340 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:35519
--webserver_interface=127.3.247.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.188.keytab
--principal=oryx/127.3.247.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:09.117539 10799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:09.117789 10799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:09.117846 10799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:09.119025 10799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:09.119092 10799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:09.119115 10799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:09.119134 10799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:09.119154 10799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:09.120678 10799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.188.keytab
--principal=oryx/127.3.247.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.247.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:09.120956 10799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:09.121183 10799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:09.125917 10804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.125927 10807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.126036 10799 server_base.cc:1034] running on GCE node
W20260806 06:51:09.125994 10805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.126334 10799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:09.126674 10799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:09.127838 10799 hybrid_clock.cc:648] HybridClock initialized: now 1785999069127827 us; error 45 us; skew 500 ppm
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:09.130481 10799 init.cc:377] Logged in from keytab as oryx/127.3.247.188@KRBTEST.COM (short username oryx)
I20260806 06:51:09.130965 10799 webserver.cc:460] Webserver started at http://127.3.247.188:38831/ using document root <none> and password file <none>
I20260806 06:51:09.131201 10799 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.131268 10799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.131426 10799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:09.132445 10799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/instance:
uuid: "84759789d77f4c468c227d234125c57e"
format_stamp: "Formatted at 2026-08-06 06:51:09 on dist-test-slave-0kls"
server_key: "c237fe4da044b626b1f3814b15010ab3"
server_key_iv: "90e33b074f5b9a795e561e9bf0735505"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.132835 10799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal/instance:
uuid: "84759789d77f4c468c227d234125c57e"
format_stamp: "Formatted at 2026-08-06 06:51:09 on dist-test-slave-0kls"
server_key: "c237fe4da044b626b1f3814b15010ab3"
server_key_iv: "90e33b074f5b9a795e561e9bf0735505"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.134142 10799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:09.134806 10814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.134960 10799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:09.135058 10799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "84759789d77f4c468c227d234125c57e"
format_stamp: "Formatted at 2026-08-06 06:51:09 on dist-test-slave-0kls"
server_key: "c237fe4da044b626b1f3814b15010ab3"
server_key_iv: "90e33b074f5b9a795e561e9bf0735505"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.135128 10799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.145265 10799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:09.147046 10799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:09.147207 10799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:09.151440 10799 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.188:35519
I20260806 06:51:09.151474 10866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.188:35519 every 8 connection(s)
I20260806 06:51:09.151847 10799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
I20260806 06:51:09.152964 10867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:09.154719 10867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.188@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:09.161933 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10799
I20260806 06:51:09.163506 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.155452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.3.247.190:36303)
0806 06:51:09.155500 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.155502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.155618 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:51:09.156044 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.156045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.156054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.156111 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.156689 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.157169 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.158189 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.158375 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.158427 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.160514 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.160515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.160515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.160523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.161935 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.162481 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.162482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.162483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.162550 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.162826 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.162827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.162828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.162958 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.163032 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.163323 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.163414 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:51:09.163707 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.155327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42398)
0806 06:51:09.155491 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.155532 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.155963 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.156181 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.156184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.156421 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.156605 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.156620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.157244 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.157245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.158114 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.158117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.158454 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.158455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.158562 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.159007 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.159016 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.160376 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.162015 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.162024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.162027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.162332 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.162637 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.162638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.162639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.162755 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.163033 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.163040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.163176 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.163288 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.163456 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260806 06:51:09.164250 10867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.188@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:09.171290 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.162503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.3.247.188:35519)
0806 06:51:09.162695 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.162707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.162721 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:51:09.162738 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.162740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.162750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.162832 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.163373 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.164421 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.165372 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.165721 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.165832 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.168265 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.168277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.168277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.168282 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.170072 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.170391 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.170394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.170394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.170453 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.170669 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.170670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.170670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.170829 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.170967 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.171089 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.171162 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260806 06:51:09.171545 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.164482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34986)
0806 06:51:09.164533 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.164542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.164569 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.164722 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.164723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.164825 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.164865 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.164865 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.165719 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.165720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.166628 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.166630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.167102 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.167103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.167168 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.167563 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.167568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.168907 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.170528 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.170528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.170529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.170731 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.170917 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.170917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.170918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.170977 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.171263 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.171264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.171300 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.171374 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.171455 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:51:09.171619 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.164563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.3.247.189:37323)
0806 06:51:09.164606 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.164608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.164615 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:09.164639 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.164640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.164653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.164715 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.164937 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.165622 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.166719 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.166993 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.167102 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.168984 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.168986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.168986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.168993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.170455 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.170795 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.170796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.170796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.170851 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.171044 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.171044 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.171045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.171173 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.171237 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.171424 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.171511 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:51:09.172011 10867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.188@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
I20260806 06:51:09.177594 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.172255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41780)
0806 06:51:09.172333 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.172351 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.172394 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.172498 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.172499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.172641 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.172675 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.172676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.173276 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.173277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.173723 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.173724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.174111 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.174112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.174160 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.174453 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.174456 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.175463 (+ 1007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.176654 (+ 1191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.176655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.176655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.176801 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.176959 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.176960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.176960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.177005 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.177240 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.177240 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.177275 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.177356 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.177486 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 06:51:09.177656 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.172303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.3.247.188:35519)
0806 06:51:09.172333 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.172334 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.172381 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:51:09.172435 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.172435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.172445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.172499 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.172746 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.173192 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.173811 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.174023 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.174111 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.175529 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.175531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.175531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.175536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.176594 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.176861 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.176862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.176862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.176905 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.177071 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.177072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.177072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.177174 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.177248 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.177378 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.177537 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":14}
I20260806 06:51:09.179082 10867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Bootstrap starting.
I20260806 06:51:09.179951 10867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:09.180259 10867 log.cc:824] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:09.180950 10867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: No bootstrap required, opened a new log
I20260806 06:51:09.182449 10867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } }
I20260806 06:51:09.182581 10867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:09.182631 10867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84759789d77f4c468c227d234125c57e, State: Initialized, Role: FOLLOWER
I20260806 06:51:09.182758 10867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } }
I20260806 06:51:09.183041 10867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:09.183007 10872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } } }
I20260806 06:51:09.183115 10872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:51:09.184602 10883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:51:09.184667 10883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:51:09.187022 10799 master_runner.cc:428] Detected that this master 84759789d77f4c468c227d234125c57e is joining an existing cluster
I20260806 06:51:09.187091 10799 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.247.188:35519
I20260806 06:51:09.194669 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41796 (local address 127.3.247.188:35519)
0806 06:51:09.188260 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.188262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.188480 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:51:09.188539 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.188539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.188550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.188604 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.189003 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.189730 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.190819 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191091 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191133 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.192299 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.192301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.192301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.192306 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.193597 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.193994 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.193994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.193995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.194037 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.194233 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.194233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.194233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.194316 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.194358 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.194550 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.194611 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 06:51:09.194785 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41796)
0806 06:51:09.188423 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.188433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.188491 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.188606 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.188615 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.188841 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.188919 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.188921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.189849 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.189850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.190732 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.190734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191202 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.191259 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.191771 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.191776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.192219 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.193675 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.193676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.193676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.193923 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.194104 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.194105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.194105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.194171 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.194372 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.194373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.194427 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.194537 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.194679 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":44,"thread_start_us":102,"threads_started":1}
I20260806 06:51:09.223122 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10209
I20260806 06:51:09.226743 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.3.247.190:36303)
0806 06:51:09.188435 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.188437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.188736 (+ 299us) server_negotiation.cc:407] Connection header received
0806 06:51:09.188832 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.188833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.188845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.188877 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.189271 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.190050 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191180 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191482 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191518 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.223477 (+ 31959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.223480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.223480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.223488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.225175 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.225770 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.225772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.225773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.225857 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.226058 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.226059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.226059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.226233 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.226280 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.226495 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.226587 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:09.227175 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42406)
0806 06:51:09.188687 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.188695 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.188760 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.188937 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.188938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.189127 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.189187 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.189189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.190135 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.190136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.191076 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.191078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191596 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.191662 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.222602 (+ 30940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.222609 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.223363 (+ 754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.225282 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.225283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.225284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.225676 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.225931 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.225931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.225932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.225993 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.226331 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.226332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.226385 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.226853 (+ 468us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.226989 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":45,"thread_start_us":61,"threads_started":1}
I20260806 06:51:09.229837 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:37119
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:36303
--webserver_port=44691
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:51:09.229867 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.3.247.189:37323)
0806 06:51:09.188303 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.188306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.188541 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:51:09.188655 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.188656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.188668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.188722 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.189221 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.189992 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191181 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191477 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191516 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.223771 (+ 32255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.223773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.223774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.223783 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.228446 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.228908 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.228910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.228910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.228976 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.229198 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.229199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.229200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.229343 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.229387 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.229644 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.229731 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 06:51:09.230711 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.188129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34992)
0806 06:51:09.188483 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.188493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.188575 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.188771 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.188771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.189022 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.189127 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.189128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.190096 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.190097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.191096 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.191097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.191596 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.191596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.191672 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.222664 (+ 30992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.222668 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.223288 (+ 620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.228548 (+ 5260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.228549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.228550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.228806 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.229052 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.229052 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.229053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.229127 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.229469 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.229469 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.229516 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.229980 (+ 464us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.230111 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":60,"spinlock_wait_cycles":9088,"thread_start_us":110,"threads_started":1}
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999068, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.190@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
I20260806 06:51:09.239303 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.231330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.3.247.188:35519)
0806 06:51:09.231368 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.231370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.231378 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.231409 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.231410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.231419 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.231473 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.231838 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.232487 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.233505 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.233863 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.233942 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.236138 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.236139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.236140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.236147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.237830 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.238276 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.238278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.238278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.238355 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.238658 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.238659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.238660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.238801 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.238869 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.239134 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.239211 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:51:09.239336 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.230799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41798)
0806 06:51:09.230845 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.230857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.230902 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.231521 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.231523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.231702 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.231750 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.231751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.232575 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.232576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.233407 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.233409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.233940 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.233942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.234013 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.234443 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.234448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.236043 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.237900 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.237901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.237902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.238179 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.238514 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.238514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.238515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.238582 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.238939 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.238940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.238991 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.239069 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.239194 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:51:09.240083 10147 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.247.188:35519: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "84759789d77f4c468c227d234125c57e"
member_type: NON_VOTER
last_known_addr {
host: "127.3.247.188"
port: 35519
}
attrs {
promote: true
}
}
dest_uuid: "675f3ce109c2436bb34fa69d23985b8c"
cas_config_opid_index: 5
I20260806 06:51:09.240322 10147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } }
I20260806 06:51:09.240792 10706 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 84759789d77f4c468c227d234125c57e (127.3.247.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.241834 10706 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.247.188:35519
I20260806 06:51:09.241907 10893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.241991 10893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.242220 10799 master.cc:537] Master@127.3.247.188:35519 shutting down...
W20260806 06:51:09.242965 10133 consensus_peers.cc:487] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c -> Peer 84759789d77f4c468c227d234125c57e (127.3.247.188:35519): Couldn't send request to peer 84759789d77f4c468c227d234125c57e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:51:09.245414 10799 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:09.245528 10799 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:09.245573 10799 tablet_replica.cc:330] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: stopping tablet replica
I20260806 06:51:09.247160 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.241983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.3.247.189:37323)
0806 06:51:09.242021 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.242023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.243380 (+ 1357us) server_negotiation.cc:407] Connection header received
0806 06:51:09.243481 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.243482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.243506 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.243543 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.243728 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.244504 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.246132 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.246732 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.246775 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.246836 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.247047 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.247092 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:09.247594 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.240899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:34994)
0806 06:51:09.243320 (+ 2421us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.243332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.243407 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.243613 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.243614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.243615 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.243650 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.243651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.244598 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.244599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.246015 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.246018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.246892 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.246893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.246955 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.246974 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.247487 (+ 513us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.247528 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2403}
I20260806 06:51:09.247975 10755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 LEARNER]: Refusing update from remote peer 675f3ce109c2436bb34fa69d23985b8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:51:09.248199 10893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 06:51:09.248817 10755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 84759789d77f4c468c227d234125c57e (127.3.247.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.249928 10895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.250006 10895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:51:09.250108 10784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675f3ce109c2436bb34fa69d23985b8c. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: NON_VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: true } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.250191 10784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:51:09.250444 10893 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: attempting to promote NON_VOTER 2b09e21849da4a27aeb9e02d6080daf3 to VOTER
I20260806 06:51:09.250614 10893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } }
I20260806 06:51:09.251026 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.250893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41814)
0806 06:51:09.250953 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.250999 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:51:09.251237 10755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 LEARNER]: Refusing update from remote peer 675f3ce109c2436bb34fa69d23985b8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 06:51:09.251430 10706 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 06:51:09.251729 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.251646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41824)
0806 06:51:09.251676 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.251704 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 06:51:09.251868 10133 consensus_peers.cc:487] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c -> Peer 84759789d77f4c468c227d234125c57e (127.3.247.188:35519): Couldn't send request to peer 84759789d77f4c468c227d234125c57e. Status: Network error: Client connection negotiation failed: client connection to 127.3.247.188:35519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:51:09.252240 10706 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.253069 10706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.253147 10706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.253191 10706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.253227 10706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.253597 10755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.254539 10784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.254614 10784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:51:09.258747 10799 master.cc:552] Master@127.3.247.188:35519 shutdown complete.
I20260806 06:51:09.258800 10799 master_runner.cc:305] Clearing existing system tablet
I20260806 06:51:09.258930 10799 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.258980 10799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.259722 10799 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:51:09.260269 10897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.260380 10799 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.260424 10799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:09.260445 10799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "84759789d77f4c468c227d234125c57e"
format_stamp: "Formatted at 2026-08-06 06:51:09 on dist-test-slave-0kls"
server_key: "c237fe4da044b626b1f3814b15010ab3"
server_key_iv: "90e33b074f5b9a795e561e9bf0735505"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.261158 10799 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:09.262655 10799 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:51:09.262720 10799 log.cc:1196] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 06:51:09.262899 10799 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Deleting consensus metadata
I20260806 06:51:09.263005 10799 master_runner.cc:315] Copying system tablet from 127.3.247.190:36303
I20260806 06:51:09.263581 10799 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: tablet copy: Beginning tablet copy session from remote peer at address 127.3.247.190:36303
I20260806 06:51:09.270932 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.263860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42410)
0806 06:51:09.263988 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.264019 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.264085 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.264219 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.264220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.264413 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.264466 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.264468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.265398 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.265399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.266297 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.266300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.266787 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.266788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.266848 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.267397 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.267403 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.267834 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.269735 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.269736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.269737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.269979 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.270240 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.270241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.270242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.270312 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.270597 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.270598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.270653 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.270730 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.270835 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260806 06:51:09.270947 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.263922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.3.247.190:36303)
0806 06:51:09.263965 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.263967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.264070 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:51:09.264137 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.264138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.264148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.264216 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.264547 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.265304 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.266378 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.266693 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.266764 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.267938 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.267940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.267940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.267947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.269646 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.270101 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.270102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.270103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.270163 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.270384 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.270385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.270385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.270521 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.270590 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.270770 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.270846 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:09.271579 10167 tablet_copy_service.cc:140] P 675f3ce109c2436bb34fa69d23985b8c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 84759789d77f4c468c227d234125c57e ({username='oryx', principal='oryx/127.3.247.188@KRBTEST.COM'} at 127.0.0.1:42410)
I20260806 06:51:09.271641 10167 tablet_copy_service.cc:161] P 675f3ce109c2436bb34fa69d23985b8c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 84759789d77f4c468c227d234125c57e at {username='oryx', principal='oryx/127.3.247.188@KRBTEST.COM'} at 127.0.0.1:42410: session id = 84759789d77f4c468c227d234125c57e-00000000000000000000000000000000
I20260806 06:51:09.272254 10167 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:51:09.272722 10799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:51:09.274035 10799 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: tablet copy: Starting download of 0 data blocks...
I20260806 06:51:09.274209 10799 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: tablet copy: Starting download of 1 WAL segments...
I20260806 06:51:09.275039 10799 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:51:09.275861 10167 tablet_copy_service.cc:342] P 675f3ce109c2436bb34fa69d23985b8c: Request end of tablet copy session 84759789d77f4c468c227d234125c57e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.247.188@KRBTEST.COM'} at 127.0.0.1:42410
I20260806 06:51:09.275911 10167 tablet_copy_service.cc:433] P 675f3ce109c2436bb34fa69d23985b8c: ending tablet copy session 84759789d77f4c468c227d234125c57e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 84759789d77f4c468c227d234125c57e
I20260806 06:51:09.278971 10799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:09.280076 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.280202 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.280615 10907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.280707 10799 server_base.cc:1034] running on GCE node
I20260806 06:51:09.280921 10799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:09.281071 10799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:09.282219 10799 hybrid_clock.cc:648] HybridClock initialized: now 1785999069282210 us; error 29 us; skew 500 ppm
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:09.283694 10799 init.cc:377] Logged in from keytab as oryx/127.3.247.188@KRBTEST.COM (short username oryx)
I20260806 06:51:09.283968 10799 webserver.cc:460] Webserver started at http://127.3.247.188:34797/ using document root <none> and password file <none>
I20260806 06:51:09.284080 10799 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.284108 10799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.285000 10799 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:09.285531 10914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.285624 10799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.285655 10799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "84759789d77f4c468c227d234125c57e"
format_stamp: "Formatted at 2026-08-06 06:51:09 on dist-test-slave-0kls"
server_key: "c237fe4da044b626b1f3814b15010ab3"
server_key_iv: "90e33b074f5b9a795e561e9bf0735505"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.285706 10799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.299199 10799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:09.301111 10799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:09.304016 10799 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.188:35519
I20260806 06:51:09.304123 10966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.188:35519 every 8 connection(s)
I20260806 06:51:09.304355 10799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
I20260806 06:51:09.306599 10967 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:51:09.307108 10967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Bootstrap starting.
I20260806 06:51:09.310103 10967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:51:09.310328 10967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: Bootstrap complete.
I20260806 06:51:09.310518 10967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } }
I20260806 06:51:09.310602 10967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 84759789d77f4c468c227d234125c57e, State: Initialized, Role: LEARNER
I20260806 06:51:09.310637 10967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } }
I20260806 06:51:09.310811 10967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:09.312216 10981 catalog_manager.cc:1259] Loaded cluster ID: 0f0abea884df4bd193d6b990f9fa8c5c
I20260806 06:51:09.312417 10981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: loading cluster ID for follower catalog manager: success
I20260806 06:51:09.313948 10981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: acquiring CA information for follower catalog manager: success
I20260806 06:51:09.314280 10981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:51:09.313911 10970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.314361 10970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: LEARNER
W20260806 06:51:09.334487 10892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:09.334733 10892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:09.334772 10892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:09.336028 10892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:09.336099 10892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:09.336146 10892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:09.337862 10892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:37119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=44691
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:09.338127 10892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:09.338364 10892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:09.343706 10987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.343698 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.343700 10986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.343930 10892 server_base.cc:1034] running on GCE node
I20260806 06:51:09.344151 10892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:09.344383 10892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:09.345518 10892 hybrid_clock.cc:648] HybridClock initialized: now 1785999069345513 us; error 35 us; skew 500 ppm
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:09.348002 10892 init.cc:377] Logged in from keytab as oryx/127.3.247.129@KRBTEST.COM (short username oryx)
I20260806 06:51:09.348424 10892 webserver.cc:460] Webserver started at http://127.3.247.129:44691/ using document root <none> and password file <none>
I20260806 06:51:09.348614 10892 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.348665 10892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.350016 10892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:09.350554 10996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.350730 10892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:09.350813 10892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "49496af4abc44429a3874ca583ee3b62"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "16654c7c27d0b04a0c154b61d44cf7c0"
server_key_iv: "80a5034bf0e206f76279f70758be532c"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.351135 10892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.367210 10892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:09.369114 10892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:09.369222 10892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:09.369402 10892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:09.369671 10892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:09.369705 10892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.369727 10892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:09.369740 10892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.375445 10892 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:37119
I20260806 06:51:09.375543 11109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:37119 every 8 connection(s)
I20260806 06:51:09.375870 10892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:09.385877 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 10892
I20260806 06:51:09.386044 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10343
I20260806 06:51:09.387317 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:52185 (local address 127.3.247.190:36303)
0806 06:51:09.376638 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.376640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.376897 (+ 257us) server_negotiation.cc:407] Connection header received
0806 06:51:09.378330 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.378331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.378345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.378377 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.379232 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.379985 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381039 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.381296 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381327 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.383558 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.383560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.383560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.383566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.385593 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386176 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.386177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.386178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.386233 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386503 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.386504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.386504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.386639 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.386696 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.387084 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.387172 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:51:09.387447 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:52185)
0806 06:51:09.376798 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.376816 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.378201 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.378461 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.378463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.378906 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.379121 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.379127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.380075 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.380076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.380947 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.380949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.381428 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.381491 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.381880 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.381885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.383469 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.385673 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.385679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.385681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.386080 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.386323 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.386324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.386441 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.386731 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.386737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.386916 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.387032 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.387211 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":33,"thread_start_us":106,"threads_started":1}
I20260806 06:51:09.387507 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:41309 (local address 127.3.247.189:37323)
0806 06:51:09.376997 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.376999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.377221 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:51:09.378350 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.378351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.378364 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.378394 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.379215 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.379706 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.380722 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.380935 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381025 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.384529 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.384530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.384531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.384537 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.386275 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386665 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.386667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.386667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.386723 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386938 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.386939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.386939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.387082 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.387141 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.387329 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.387401 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:09.387882 11111 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:09.387984 11111 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.388074 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.3.247.129:41309)
0806 06:51:09.377155 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.377164 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.378216 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.378505 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.378506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.378899 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.379121 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.379125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.379817 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.379818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.380631 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.380633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.381034 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.381185 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.381736 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.381747 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.384458 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.386363 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.386364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.386578 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.386801 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.386802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.386802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.386869 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.387168 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.387169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.387207 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.387830 (+ 623us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.387925 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260806 06:51:09.388195 11111 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:09.388595 10147 ts_manager.cc:194] Re-registered known tserver with Master: 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119)
I20260806 06:51:09.389142 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:52185
I20260806 06:51:09.389374 11112 heartbeater.cc:347] Connected to a master server at 127.3.247.189:37323
I20260806 06:51:09.389429 11112 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.390081 11112 heartbeater.cc:511] Master 127.3.247.189:37323 requested a full tablet report, sending...
I20260806 06:51:09.390455 10745 ts_manager.cc:194] Registered new tserver with Master: 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119)
I20260806 06:51:09.392633 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:36913
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:36303
--webserver_port=38381
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:51:09.393854 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:54575 (local address 127.3.247.188:35519)
0806 06:51:09.376704 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.376707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.376896 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:51:09.378299 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.378300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.378316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.378375 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.379233 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.380241 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381164 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.381463 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381564 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.384047 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.384048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.384048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.384056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.392622 (+ 8566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.392995 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.392997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.392997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.393065 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.393288 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.393289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.393289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.393408 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.393484 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.393650 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.393736 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":51,"threads_started":1}
I20260806 06:51:09.394016 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.376586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.3.247.129:54575)
0806 06:51:09.376770 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.376817 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.378201 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.378459 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.378463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.378914 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.379121 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.379127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.380325 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.380326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.381077 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.381079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.381579 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.381580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.381683 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.382051 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.382055 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.383926 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.392723 (+ 8797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.392723 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.392724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.392905 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.393155 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.393156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.393157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.393222 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.393475 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.393476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.393516 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.393620 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.393733 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":33,"thread_start_us":86,"threads_started":1}
I20260806 06:51:09.394420 11110 heartbeater.cc:347] Connected to a master server at 127.3.247.188:35519
I20260806 06:51:09.394472 11110 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.394513 11110 heartbeater.cc:511] Master 127.3.247.188:35519 requested a full tablet report, sending...
I20260806 06:51:09.394840 10931 ts_manager.cc:194] Registered new tserver with Master: 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119)
W20260806 06:51:09.473405 11118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:09.473603 11118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:09.473625 11118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:09.474758 11118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:09.474799 11118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:09.474825 11118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:09.476202 11118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:36913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=38381
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:09.476418 11118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:09.476681 11118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:09.481698 11123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.481727 11124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.481727 11126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.481796 11118 server_base.cc:1034] running on GCE node
I20260806 06:51:09.482062 11118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:09.482298 11118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:09.483446 11118 hybrid_clock.cc:648] HybridClock initialized: now 1785999069483428 us; error 33 us; skew 500 ppm
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:09.485952 11118 init.cc:377] Logged in from keytab as oryx/127.3.247.130@KRBTEST.COM (short username oryx)
I20260806 06:51:09.486409 11118 webserver.cc:460] Webserver started at http://127.3.247.130:38381/ using document root <none> and password file <none>
I20260806 06:51:09.486617 11118 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.486671 11118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.487890 11118 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:09.488449 11133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.488638 11118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:09.488729 11118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "495bed5977624d109524bc9811388206"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "f2113de092ab0f293a81576045a82ecf"
server_key_iv: "7dc482bbdad066d03f9435d76b805373"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.489053 11118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.510039 11118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:09.512090 11118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:09.512265 11118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:09.512495 11118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:09.512827 11118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:09.512883 11118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.512924 11118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:09.512953 11118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.518774 11118 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:36913
I20260806 06:51:09.518824 11246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:36913 every 8 connection(s)
I20260806 06:51:09.519176 11118 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:09.527603 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11118
I20260806 06:51:09.527765 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10477
I20260806 06:51:09.530695 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.519882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:39573 (local address 127.3.247.190:36303)
0806 06:51:09.519936 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.519939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.520598 (+ 659us) server_negotiation.cc:407] Connection header received
0806 06:51:09.520890 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.520891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.520904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.520986 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.521855 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522356 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.523480 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.523719 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.523781 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.526656 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.526657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.526657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.526666 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.528494 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.529423 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.529425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.529425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.529488 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.529814 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.529815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.529816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.529969 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.530015 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.530430 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.530518 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:51:09.531306 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.519890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:39573)
0806 06:51:09.520503 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.520512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.520782 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.521068 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.521072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.521527 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.521728 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.521732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522461 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.522463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.523383 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.523385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.523841 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.523842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.523978 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.524494 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.524502 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.526550 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.528607 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.528613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.528616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.529322 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.529561 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.529562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.529562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.529713 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.530062 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.530067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.530233 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.530822 (+ 589us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.531015 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":464,"thread_start_us":97,"threads_started":1}
I20260806 06:51:09.531352 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.520342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:42625 (local address 127.3.247.188:35519)
0806 06:51:09.520391 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.520394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.520446 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:51:09.520891 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.520893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.520908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.520939 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.521857 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522622 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.523751 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.523971 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.524050 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.526185 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.526186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.526187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.526193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.528261 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530123 (+ 1862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.530124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.530124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.530180 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530395 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.530396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.530397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.530527 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.530600 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.531154 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.531249 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:51:09.531378 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.519922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:50429 (local address 127.3.247.189:37323)
0806 06:51:09.520011 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.520013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.520187 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:51:09.520891 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.520892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.520905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.520939 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.521856 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522637 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.523725 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.524013 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.524079 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.527188 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.527190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.527190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.527198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.530034 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530487 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.530488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.530488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.530539 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530838 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.530839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.530839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.530975 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.531015 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.531209 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.531293 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 06:51:09.531857 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.519890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.3.247.130:50429)
0806 06:51:09.520053 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.520093 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.520769 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.521069 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.521073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.521552 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.521729 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.521735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522725 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.522726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.523645 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.523647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.524133 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.524134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.524216 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.524834 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.524838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.527100 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.530111 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.530113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.530368 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.530631 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.530632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.530687 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.531060 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.531061 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.531100 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.531653 (+ 553us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.531753 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":75,"thread_start_us":89,"threads_started":1}
I20260806 06:51:09.531942 11248 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:09.532023 11248 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.532229 11248 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:09.532608 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.520239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.3.247.130:42625)
0806 06:51:09.520387 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.520398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.520773 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.521069 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.521073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.521545 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.521729 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.521735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.522725 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.522726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.523648 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.523650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.524047 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.524048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.524111 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.524646 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.524650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.526094 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.528621 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.528622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.528623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.530005 (+ 1382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.530267 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.530268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.530268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.530323 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.530641 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.530642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.530689 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.532400 (+ 1711us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.532482 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":50,"thread_start_us":92,"threads_started":1}
I20260806 06:51:09.532697 10147 ts_manager.cc:194] Re-registered known tserver with Master: 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:09.533224 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:39573
I20260806 06:51:09.535001 11247 heartbeater.cc:347] Connected to a master server at 127.3.247.188:35519
I20260806 06:51:09.535063 11247 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.535137 11247 heartbeater.cc:511] Master 127.3.247.188:35519 requested a full tablet report, sending...
I20260806 06:51:09.535176 11249 heartbeater.cc:347] Connected to a master server at 127.3.247.189:37323
I20260806 06:51:09.535235 11249 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.535292 11249 heartbeater.cc:511] Master 127.3.247.189:37323 requested a full tablet report, sending...
I20260806 06:51:09.535454 10931 ts_manager.cc:194] Registered new tserver with Master: 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:09.535583 10745 ts_manager.cc:194] Registered new tserver with Master: 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:09.535943 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:45657
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:36303
--webserver_port=45371
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:09.615899 11254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:09.616176 11254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:09.616216 11254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:09.617394 11254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:09.617457 11254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:09.617502 11254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:09.618991 11254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:45657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=45371
--tserver_master_addrs=127.3.247.190:36303,127.3.247.189:37323,127.3.247.188:35519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:09.619277 11254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:09.619509 11254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:09.624523 11260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.624500 11259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:09.624481 11262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:09.624528 11254 server_base.cc:1034] running on GCE node
I20260806 06:51:09.624967 11254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:09.625262 11254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:09.626423 11254 hybrid_clock.cc:648] HybridClock initialized: now 1785999069626408 us; error 26 us; skew 500 ppm
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:09.628961 11254 init.cc:377] Logged in from keytab as oryx/127.3.247.131@KRBTEST.COM (short username oryx)
I20260806 06:51:09.629452 11254 webserver.cc:460] Webserver started at http://127.3.247.131:45371/ using document root <none> and password file <none>
I20260806 06:51:09.629681 11254 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:09.629763 11254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:09.631237 11254 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:09.631817 11269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.631980 11254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:09.632092 11254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "6ca86b739b714c618525f500c7c91ac1"
format_stamp: "Formatted at 2026-08-06 06:51:08 on dist-test-slave-0kls"
server_key: "e9b8b1af81819af30d7d2a3bf2a7d85e"
server_key_iv: "145415ae7b6be40e3099f80c7a824927"
server_key_version: "encryptionkey@0"
I20260806 06:51:09.632429 11254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:09.653967 11254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:09.656386 11254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:09.656565 11254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:09.656811 11254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:09.657148 11254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:09.657207 11254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.657243 11254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:09.657269 11254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:09.663113 11254 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:45657
I20260806 06:51:09.663154 11382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:45657 every 8 connection(s)
I20260806 06:51:09.663520 11254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:09.670662 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11254
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.188@KRBTEST.COM
Aug 06 06:51:09 dist-test-slave-0kls krb5kdc[10102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999069, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.189@KRBTEST.COM
I20260806 06:51:09.674937 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:33653 (local address 127.3.247.190:36303)
0806 06:51:09.664576 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.664578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.664637 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:51:09.665233 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.665234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.665246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.665297 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.666263 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.666790 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668240 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668533 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668609 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.670923 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.670924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.670924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.670935 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.673489 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673913 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.673914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.673915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.673992 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674199 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.674200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.674200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.674354 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.674422 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.674757 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.674846 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:51:09.674985 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:49493 (local address 127.3.247.188:35519)
0806 06:51:09.664686 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.664688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.664697 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:09.665237 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.665239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.665251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.665284 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.666869 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.667322 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668110 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668384 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668449 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.671393 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.671394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.671395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.671403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.673225 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673730 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.673731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.673732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.673817 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674140 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.674141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.674141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.674279 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.674326 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.674777 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.674867 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:09.675580 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36101 (local address 127.3.247.189:37323)
0806 06:51:09.664573 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.664575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.664833 (+ 258us) server_negotiation.cc:407] Connection header received
0806 06:51:09.665238 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.665239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.665252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.665284 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.666233 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.666813 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668269 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668516 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668579 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.671982 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.671984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.671984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.671992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.673664 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674070 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.674071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.674071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.674132 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674328 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.674328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.674329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.674480 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.674537 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.675397 (+ 860us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.675459 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:51:09.675843 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.3.247.131:36101)
0806 06:51:09.664751 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.664760 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.665140 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.665802 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.665803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.665951 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.666101 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.666102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.666919 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.666920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668156 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.668158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668633 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668686 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.669342 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.669347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.671886 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.673757 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.673758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.673996 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.674199 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.674200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.674263 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.674560 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.674560 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.674592 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.674701 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.674872 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":73,"thread_start_us":66,"threads_started":1}
I20260806 06:51:09.675843 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:33653)
0806 06:51:09.664455 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.664498 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.665118 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.665372 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.665375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.665862 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.666081 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.666092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.666892 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.666893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668157 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.668158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668608 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668696 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.669280 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.669288 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.670817 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.673578 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.673580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.673825 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.674066 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.674067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.674068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.674124 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.674448 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.674448 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.674598 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.675074 (+ 476us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.675189 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":99,"thread_start_us":111,"threads_started":1}
I20260806 06:51:09.676293 11384 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36303
I20260806 06:51:09.676378 11384 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.676569 11384 heartbeater.cc:511] Master 127.3.247.190:36303 requested a full tablet report, sending...
I20260806 06:51:09.676956 10147 ts_manager.cc:194] Re-registered known tserver with Master: 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:09.677382 10147 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:33653
I20260806 06:51:09.677419 11385 heartbeater.cc:347] Connected to a master server at 127.3.247.189:37323
I20260806 06:51:09.677465 11385 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.677536 11385 heartbeater.cc:511] Master 127.3.247.189:37323 requested a full tablet report, sending...
I20260806 06:51:09.677865 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.664261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.3.247.131:49493)
0806 06:51:09.664625 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.664636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.665140 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.665374 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.665377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.665855 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:09.666099 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.666102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.667405 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.667407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668019 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.668020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.668469 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.668470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.668615 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.669461 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:09.669465 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:09.671260 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:09.673318 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.673328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.673648 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.673898 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:09.673900 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:09.673900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:09.674059 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:09.674376 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:09.674400 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:09.674590 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.676256 (+ 1666us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.676355 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":28,"thread_start_us":94,"threads_started":1}
I20260806 06:51:09.677871 10745 ts_manager.cc:194] Registered new tserver with Master: 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:09.678238 11383 heartbeater.cc:347] Connected to a master server at 127.3.247.188:35519
I20260806 06:51:09.678280 11383 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:09.678339 11383 heartbeater.cc:511] Master 127.3.247.188:35519 requested a full tablet report, sending...
I20260806 06:51:09.678582 10931 ts_manager.cc:194] Registered new tserver with Master: 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:09.684561 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.672398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.3.247.188:35519)
0806 06:51:09.675058 (+ 2660us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.675060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.675068 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.675087 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.675088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.675097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.675150 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.678055 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.678787 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.679811 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.680113 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.680185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.681618 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.681619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.681620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.681625 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.683230 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.683621 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.683622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.683623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.683683 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.683930 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.683931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.683931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.684078 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.684181 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.684368 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.684448 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2645,"thread_start_us":46,"threads_started":1}
I20260806 06:51:09.685668 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.673867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.3.247.189:37323)
0806 06:51:09.674725 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.674727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.674734 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:09.674753 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.674754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.674764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.674818 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.676409 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.679849 (+ 3440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.680805 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.680996 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.681049 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.682406 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.682407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.682407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.682414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.684260 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.684701 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.684702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.684703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.684870 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.685144 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.685145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.685145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.685252 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.685330 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.685450 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.685536 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":69,"threads_started":1}
I20260806 06:51:09.686441 11398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.674242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.3.247.190:36303)
0806 06:51:09.674526 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.674527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.675520 (+ 993us) server_negotiation.cc:407] Connection header received
0806 06:51:09.675562 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.675563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.675576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.675701 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.678825 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.679580 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.680488 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.681850 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.681925 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.683235 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.683236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.683237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.683244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.685075 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.685636 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.685637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.685637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.685694 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.685939 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.685940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.685940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.686050 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.686165 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.686259 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.686340 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":79,"threads_started":1}
I20260806 06:51:09.687939 10147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:09.689458 10147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:09.696715 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37119 (local address 127.0.0.1:53066)
0806 06:51:09.693153 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.693164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.693207 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.693540 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.693541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.693543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.693595 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.693597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.694703 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.694703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.695551 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.695553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.696465 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.696528 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.696532 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.696589 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.696638 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 06:51:09.696734 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53066 (local address 127.3.247.129:37119)
0806 06:51:09.693350 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.693355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.693365 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:09.693413 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.693415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.693433 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.693690 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.693763 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.694602 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.695639 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.696378 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696462 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.696546 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.696605 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.696646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
I20260806 06:51:09.697248 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:45657 (local address 127.0.0.1:39726)
0806 06:51:09.693716 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.693726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.693777 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.693974 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.693974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.693977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.694016 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.694017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.695236 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.695237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.696065 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.696067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.696997 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.697062 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.697066 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.697131 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.697170 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":14,"thread_start_us":69,"threads_started":1}
I20260806 06:51:09.697270 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.3.247.131:45657)
0806 06:51:09.693536 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.693541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.693824 (+ 283us) server_negotiation.cc:407] Connection header received
0806 06:51:09.693869 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.693871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.693886 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.693967 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.694095 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.695147 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696150 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.696763 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696983 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.697066 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.697155 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.697194 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260806 06:51:09.697633 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:36913 (local address 127.0.0.1:35008)
0806 06:51:09.693585 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.693595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.693638 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.693927 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.693928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.693930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.693976 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.693978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.695265 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.695266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.696410 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.696413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.697403 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.697404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.697464 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.697469 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.697521 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.697561 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":97,"threads_started":1}
I20260806 06:51:09.697747 11044 tablet_service.cc:1467] Processing CreateTablet for tablet 6c788486e184473eb1215f80ba0aab5c (DEFAULT_TABLE table=test-table [id=7a051514cdb04e63a930e2722491d9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:09.698167 11044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c788486e184473eb1215f80ba0aab5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:09.698077 11317 tablet_service.cc:1467] Processing CreateTablet for tablet 6c788486e184473eb1215f80ba0aab5c (DEFAULT_TABLE table=test-table [id=7a051514cdb04e63a930e2722491d9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:09.698380 11317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c788486e184473eb1215f80ba0aab5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:09.701857 11409 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Bootstrap starting.
I20260806 06:51:09.702020 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.693321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.3.247.130:36913)
0806 06:51:09.693647 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.693652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.693661 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:09.693709 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.693710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.693722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.693853 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.694050 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.695144 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.696500 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.697232 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.697295 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.697369 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.697555 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.697601 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":77,"threads_started":1}
I20260806 06:51:09.702600 11409 tablet_bootstrap.cc:654] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:09.702759 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 6c788486e184473eb1215f80ba0aab5c (DEFAULT_TABLE table=test-table [id=7a051514cdb04e63a930e2722491d9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:09.702907 11409 log.cc:824] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:09.703094 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c788486e184473eb1215f80ba0aab5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:09.703300 11410 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Bootstrap starting.
I20260806 06:51:09.703840 11410 tablet_bootstrap.cc:654] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:09.704160 11410 log.cc:824] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:09.704680 11409 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: No bootstrap required, opened a new log
I20260806 06:51:09.704757 11409 ts_tablet_manager.cc:1397] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:51:09.704955 11410 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: No bootstrap required, opened a new log
I20260806 06:51:09.705016 11410 ts_tablet_manager.cc:1397] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:09.706857 11413 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Bootstrap starting.
I20260806 06:51:09.707202 11410 raft_consensus.cc:348] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.707363 11410 raft_consensus.cc:374] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:09.707394 11410 raft_consensus.cc:729] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49496af4abc44429a3874ca583ee3b62, State: Initialized, Role: FOLLOWER
I20260806 06:51:09.707448 11413 tablet_bootstrap.cc:654] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:09.707517 11410 consensus_queue.cc:260] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.707573 11409 raft_consensus.cc:348] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.707734 11409 raft_consensus.cc:374] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:09.707767 11409 raft_consensus.cc:729] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ca86b739b714c618525f500c7c91ac1, State: Initialized, Role: FOLLOWER
I20260806 06:51:09.707769 11413 log.cc:824] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:09.707783 11410 ts_tablet_manager.cc:1428] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:09.707886 11111 heartbeater.cc:503] Master 127.3.247.190:36303 was elected leader, sending a full tablet report...
I20260806 06:51:09.708711 11409 consensus_queue.cc:260] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.708915 11413 tablet_bootstrap.cc:492] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: No bootstrap required, opened a new log
I20260806 06:51:09.708966 11413 ts_tablet_manager.cc:1397] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:09.709146 11384 heartbeater.cc:503] Master 127.3.247.190:36303 was elected leader, sending a full tablet report...
I20260806 06:51:09.709640 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.705560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:41852)
0806 06:51:09.705619 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.705631 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.705681 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.705933 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.705935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.705937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.705982 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.705984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.706876 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.706877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.708456 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.708459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.709390 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.709391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.709457 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.709462 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.709513 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.709561 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:51:09.709661 11409 ts_tablet_manager.cc:1428] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 06:51:09.709867 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.705670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.3.247.188:35519)
0806 06:51:09.705715 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.705717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.705725 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.705751 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.705752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.705774 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.705881 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.706059 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.706787 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.708539 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.709140 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.709210 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.709280 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.709730 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.709780 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:51:09.711395 11413 raft_consensus.cc:348] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.711551 11413 raft_consensus.cc:374] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:09.711597 11413 raft_consensus.cc:729] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495bed5977624d109524bc9811388206, State: Initialized, Role: FOLLOWER
I20260806 06:51:09.711735 11413 consensus_queue.cc:260] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.711942 11413 ts_tablet_manager.cc:1428] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:51:09.712002 11248 heartbeater.cc:503] Master 127.3.247.190:36303 was elected leader, sending a full tablet report...
I20260806 06:51:09.712579 10941 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 06:51:09.713392 10970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675f3ce109c2436bb34fa69d23985b8c. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.713478 10970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: LEARNER
I20260806 06:51:09.752187 11414 raft_consensus.cc:482] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:09.752305 11414 raft_consensus.cc:504] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.752853 11414 leader_election.cc:288] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 495bed5977624d109524bc9811388206 (127.3.247.130:36913), 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:09.755963 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.753166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:42191 (local address 127.3.247.131:45657)
0806 06:51:09.753254 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.753256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.753264 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.753289 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.753290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.753299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.753349 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.753621 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.754345 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755260 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.755620 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755714 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.755723 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.755853 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.755891 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 06:51:09.755963 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.753026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:45657 (local address 127.3.247.129:42191)
0806 06:51:09.753101 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.753114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.753166 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.753502 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.753503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.753504 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.753554 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.753555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.754410 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.754411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.755175 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.755178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.755713 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.755782 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.755784 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.755847 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.755891 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:51:09.756026 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.753140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:55471 (local address 127.3.247.130:36913)
0806 06:51:09.753202 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.753204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.753212 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.753240 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.753241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.753250 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.753307 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:09.753492 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.754273 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755268 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.755658 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755739 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.755747 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.755920 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.755959 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:51:09.756078 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.753045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:36913 (local address 127.3.247.129:55471)
0806 06:51:09.753093 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:09.753104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:09.753161 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:09.753336 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:09.753337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:09.753339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:09.753417 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.753419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.754348 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.754349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.755178 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:09.755180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.755749 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.755750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:09.755835 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.755838 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:09.755911 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:51:09.755952 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 06:51:09.756309 11337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c788486e184473eb1215f80ba0aab5c" candidate_uuid: "49496af4abc44429a3874ca583ee3b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ca86b739b714c618525f500c7c91ac1" is_pre_election: true
I20260806 06:51:09.756419 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c788486e184473eb1215f80ba0aab5c" candidate_uuid: "49496af4abc44429a3874ca583ee3b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495bed5977624d109524bc9811388206" is_pre_election: true
I20260806 06:51:09.756472 11337 raft_consensus.cc:2454] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49496af4abc44429a3874ca583ee3b62 in term 0.
I20260806 06:51:09.756584 11201 raft_consensus.cc:2454] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49496af4abc44429a3874ca583ee3b62 in term 0.
I20260806 06:51:09.756733 10999 leader_election.cc:302] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49496af4abc44429a3874ca583ee3b62, 6ca86b739b714c618525f500c7c91ac1; no voters:
I20260806 06:51:09.756876 11414 raft_consensus.cc:2781] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:09.756930 11414 raft_consensus.cc:482] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:09.756953 11414 raft_consensus.cc:3037] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:09.757611 11414 raft_consensus.cc:504] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.757755 11414 leader_election.cc:288] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [CANDIDATE]: Term 1 election: Requested vote from peers 495bed5977624d109524bc9811388206 (127.3.247.130:36913), 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657)
I20260806 06:51:09.757941 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c788486e184473eb1215f80ba0aab5c" candidate_uuid: "49496af4abc44429a3874ca583ee3b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495bed5977624d109524bc9811388206"
I20260806 06:51:09.757949 11337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c788486e184473eb1215f80ba0aab5c" candidate_uuid: "49496af4abc44429a3874ca583ee3b62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ca86b739b714c618525f500c7c91ac1"
I20260806 06:51:09.758021 11337 raft_consensus.cc:3037] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:09.758035 11201 raft_consensus.cc:3037] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:09.758889 11337 raft_consensus.cc:2454] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49496af4abc44429a3874ca583ee3b62 in term 1.
I20260806 06:51:09.758889 11201 raft_consensus.cc:2454] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49496af4abc44429a3874ca583ee3b62 in term 1.
I20260806 06:51:09.759047 10999 leader_election.cc:302] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49496af4abc44429a3874ca583ee3b62, 6ca86b739b714c618525f500c7c91ac1; no voters:
I20260806 06:51:09.759148 11414 raft_consensus.cc:2781] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:09.759292 11414 raft_consensus.cc:686] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 1 LEADER]: Becoming Leader. State: Replica: 49496af4abc44429a3874ca583ee3b62, State: Running, Role: LEADER
I20260806 06:51:09.759393 11414 consensus_queue.cc:237] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } }
I20260806 06:51:09.760202 10147 catalog_manager.cc:5495] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49496af4abc44429a3874ca583ee3b62 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "49496af4abc44429a3874ca583ee3b62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } health_report { overall_health: UNKNOWN } } }
I20260806 06:51:09.760797 10893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: NON_VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: true } } }
I20260806 06:51:09.760903 10893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.761431 10893 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: attempting to promote NON_VOTER 84759789d77f4c468c227d234125c57e to VOTER
I20260806 06:51:09.761572 10893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:09.761938 10755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Refusing update from remote peer 675f3ce109c2436bb34fa69d23985b8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:51:09.762007 10940 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 LEARNER]: Refusing update from remote peer 675f3ce109c2436bb34fa69d23985b8c: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:51:09.762153 10893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:51:09.762264 10893 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260806 06:51:09.762854 11424 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 84759789d77f4c468c227d234125c57e (127.3.247.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.762998 10755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 84759789d77f4c468c227d234125c57e (127.3.247.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.763077 10941 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 84759789d77f4c468c227d234125c57e (127.3.247.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.763844 11424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.763933 11424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.763892 10707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.763976 10707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.763954 10970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.763984 11424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.764025 10970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:51:09.764042 11424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:09.764123 11426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675f3ce109c2436bb34fa69d23985b8c" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:09.764207 11426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:51:09.768966 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.766607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.3.247.129:37119)
0806 06:51:09.766654 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.766656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.766664 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:09.766698 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.766699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.766717 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.766785 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:09.766944 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.767413 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.768166 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.768372 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.768456 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.768630 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:09.768749 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:09.768834 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.768871 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.768891 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 06:51:09.769925 11250 tablet.cc:2369] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:09.770690 11337 raft_consensus.cc:1261] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Refusing update from remote peer 49496af4abc44429a3874ca583ee3b62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:09.770706 11201 raft_consensus.cc:1261] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Refusing update from remote peer 49496af4abc44429a3874ca583ee3b62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:09.771018 11414 consensus_queue.cc:1035] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:09.771142 11422 consensus_queue.cc:1035] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:09.773411 11428 mvcc.cc:205] Tried to move back new op lower bound from 7315452189778157568 to 7315452189735071744. Current Snapshot: MvccSnapshot[applied={T|T < 7315452189778157568}]
I20260806 06:51:09.774379 11429 mvcc.cc:205] Tried to move back new op lower bound from 7315452189778157568 to 7315452189735071744. Current Snapshot: MvccSnapshot[applied={T|T < 7315452189778157568}]
I20260806 06:51:09.775260 11430 mvcc.cc:205] Tried to move back new op lower bound from 7315452189778157568 to 7315452189735071744. Current Snapshot: MvccSnapshot[applied={T|T < 7315452189778157568 or (T in {7315452189778157568})}]
I20260806 06:51:09.777118 10146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:09.777230 10146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:09.778545 10146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 675f3ce109c2436bb34fa69d23985b8c: Sending DeleteTablet for 3 replicas of tablet 6c788486e184473eb1215f80ba0aab5c
I20260806 06:51:09.778832 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c788486e184473eb1215f80ba0aab5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:09 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:35008
I20260806 06:51:09.779028 11438 tablet_replica.cc:330] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: stopping tablet replica
I20260806 06:51:09.779175 11438 raft_consensus.cc:2229] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:09.779367 11438 raft_consensus.cc:2258] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:09.779747 11317 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c788486e184473eb1215f80ba0aab5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:09 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:39726
I20260806 06:51:09.779870 11044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c788486e184473eb1215f80ba0aab5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:09 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:53066
I20260806 06:51:09.779935 11440 tablet_replica.cc:330] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: stopping tablet replica
I20260806 06:51:09.780019 11440 raft_consensus.cc:2229] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:09.780120 11440 raft_consensus.cc:2258] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:09.780335 11438 ts_tablet_manager.cc:1905] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:09.780541 11440 ts_tablet_manager.cc:1905] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:09.780694 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10115
I20260806 06:51:09.782721 11441 tablet_replica.cc:330] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: stopping tablet replica
I20260806 06:51:09.782815 11441 raft_consensus.cc:2229] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:09.782922 11441 raft_consensus.cc:2258] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:09.783087 11440 ts_tablet_manager.cc:1918] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:09.783142 11440 log.cc:1196] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/6c788486e184473eb1215f80ba0aab5c
I20260806 06:51:09.783404 11440 ts_tablet_manager.cc:1939] T 6c788486e184473eb1215f80ba0aab5c P 6ca86b739b714c618525f500c7c91ac1: Deleting consensus metadata
I20260806 06:51:09.783860 11438 ts_tablet_manager.cc:1918] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:09.783931 11438 log.cc:1196] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/6c788486e184473eb1215f80ba0aab5c
I20260806 06:51:09.784186 11438 ts_tablet_manager.cc:1939] T 6c788486e184473eb1215f80ba0aab5c P 495bed5977624d109524bc9811388206: Deleting consensus metadata
I20260806 06:51:09.784726 11441 ts_tablet_manager.cc:1905] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:09.786432 11441 ts_tablet_manager.cc:1918] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:09.786504 11441 log.cc:1196] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/6c788486e184473eb1215f80ba0aab5c
I20260806 06:51:09.786844 11441 ts_tablet_manager.cc:1939] T 6c788486e184473eb1215f80ba0aab5c P 49496af4abc44429a3874ca583ee3b62: Deleting consensus metadata
W20260806 06:51:09.787992 11273 connection.cc:535] server connection from 127.0.0.1:39726 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:51:09.788017 11137 connection.cc:535] server connection from 127.0.0.1:35008 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:51:09.788244 11000 connection.cc:535] server connection from 127.0.0.1:53066 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:51:09.790750 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.779415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.3.247.188:35519)
0806 06:51:09.779452 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.779454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.779461 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:09.779481 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.779482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.779490 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.779545 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.782667 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.783436 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.785016 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.785299 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.785344 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.787497 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.787499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.787499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.787507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.789421 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.789848 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.789850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.789850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.789920 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.790119 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.790120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.790121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.790275 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.790376 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.790572 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.790648 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:09.797676 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.790543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.3.247.188:35519)
0806 06:51:09.790736 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.790738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.790745 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:09.790763 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.790763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.790773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.790828 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.791757 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.792240 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.793092 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.793305 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.793389 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.794980 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.794981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.794981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.794988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.796500 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.796891 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.796892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.796892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.796948 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.797168 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.797169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.797170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.797283 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.797386 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.797486 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.797567 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260806 06:51:09.798811 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:09.790502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.3.247.189:37323)
0806 06:51:09.790583 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:09.790585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:09.790594 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:09.790618 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:09.790619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:09.790631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:09.790687 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:09.791968 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.792771 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.794005 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:09.794325 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:09.794405 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:09.795727 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:09.795728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:09.795728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:09.795735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:09.797282 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.797668 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.797669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.797670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.797988 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:09.798265 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:09.798267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:09.798267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:09.798383 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:09.798501 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:09.798593 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:51:09.798696 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 06:51:10.775486 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:10.775203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:58357)
0806 06:51:10.775345 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:10.775430 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
I20260806 06:51:10.775686 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:10.775450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:40043)
0806 06:51:10.775558 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:10.775627 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
I20260806 06:51:10.775789 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:10.775458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:39079)
0806 06:51:10.775683 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:10.775744 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":46,"threads_started":1}
W20260806 06:51:10.775856 11111 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:36303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
W20260806 06:51:10.775859 11248 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:36303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
W20260806 06:51:10.776041 11384 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:36303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.279088 11452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 675f3ce109c2436bb34fa69d23985b8c)
I20260806 06:51:11.279215 11452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:11.279438 11452 leader_election.cc:288] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 84759789d77f4c468c227d234125c57e (127.3.247.188:35519)
I20260806 06:51:11.279675 11453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 675f3ce109c2436bb34fa69d23985b8c)
I20260806 06:51:11.279826 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.279585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41744)
0806 06:51:11.279701 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.279769 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260806 06:51:11.279798 11453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
W20260806 06:51:11.279994 10732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.280086 11453 leader_election.cc:288] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189:37323)
I20260806 06:51:11.280437 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.280181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41752)
0806 06:51:11.280345 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.280389 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260806 06:51:11.280632 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.280523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41764)
0806 06:51:11.280562 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.280590 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:51:11.280644 10918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
W20260806 06:51:11.280946 10732 leader_election.cc:334] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.281040 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.280964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41778)
0806 06:51:11.280998 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.281018 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:51:11.281107 10918 leader_election.cc:334] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.283214 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.279590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:35519 (local address 127.0.0.1:53438)
0806 06:51:11.279700 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.279713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:11.279770 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:11.280308 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:11.280309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:11.280311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:11.280377 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:11.280380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.281329 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.281330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:11.282273 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:11.282275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.282942 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.282943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:11.283001 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:11.283005 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:11.283065 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:51:11.283107 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":48,"threads_started":1}
I20260806 06:51:11.283190 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.279722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.3.247.188:35519)
0806 06:51:11.279867 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:11.279870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:11.279878 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:11.280191 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:11.280193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:11.280207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:11.280443 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:11.280479 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.281253 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.282350 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.282876 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.282926 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:11.282935 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:11.283097 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:51:11.283142 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":44,"threads_started":1}
I20260806 06:51:11.283541 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.280283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.3.247.189:37323)
0806 06:51:11.280406 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:11.280408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:11.280519 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:51:11.280620 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:11.280621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:11.280642 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:11.280686 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:11.280883 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.281618 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.282725 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.283172 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.283238 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:11.283249 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:11.283434 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 06:51:11.283480 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260806 06:51:11.283536 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.280227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:37323 (local address 127.0.0.1:50284)
0806 06:51:11.280454 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.280465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:11.280544 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:11.280752 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:11.280753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:11.280755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:11.280806 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:11.280808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.281705 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.281706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:11.282654 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:11.282656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:11.283265 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:11.283266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:11.283340 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:11.283345 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:11.283419 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:51:11.283468 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":47,"threads_started":1}
I20260806 06:51:11.283601 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b09e21849da4a27aeb9e02d6080daf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "84759789d77f4c468c227d234125c57e" is_pre_election: true
I20260806 06:51:11.283715 10941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b09e21849da4a27aeb9e02d6080daf3 in term 1.
I20260806 06:51:11.283787 10755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84759789d77f4c468c227d234125c57e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b09e21849da4a27aeb9e02d6080daf3" is_pre_election: true
I20260806 06:51:11.283865 10731 leader_election.cc:302] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b09e21849da4a27aeb9e02d6080daf3, 84759789d77f4c468c227d234125c57e; no voters: 675f3ce109c2436bb34fa69d23985b8c
I20260806 06:51:11.283918 10755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84759789d77f4c468c227d234125c57e in term 1.
I20260806 06:51:11.283965 11452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:51:11.284003 11452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 675f3ce109c2436bb34fa69d23985b8c)
I20260806 06:51:11.284029 11452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:51:11.284077 10915 leader_election.cc:302] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b09e21849da4a27aeb9e02d6080daf3, 84759789d77f4c468c227d234125c57e; no voters: 675f3ce109c2436bb34fa69d23985b8c
I20260806 06:51:11.284184 11453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:51:11.284229 11453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 675f3ce109c2436bb34fa69d23985b8c)
I20260806 06:51:11.284245 11453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:51:11.284791 11452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:11.284911 11452 leader_election.cc:288] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 election: Requested vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 84759789d77f4c468c227d234125c57e (127.3.247.188:35519)
I20260806 06:51:11.284868 11453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:11.285015 11453 leader_election.cc:288] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 election: Requested vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 2b09e21849da4a27aeb9e02d6080daf3 (127.3.247.189:37323)
I20260806 06:51:11.285115 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.285011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41780)
0806 06:51:11.285055 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.285083 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:51:11.285209 10755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84759789d77f4c468c227d234125c57e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b09e21849da4a27aeb9e02d6080daf3"
I20260806 06:51:11.285221 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b09e21849da4a27aeb9e02d6080daf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "84759789d77f4c468c227d234125c57e"
I20260806 06:51:11.285298 10941 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2b09e21849da4a27aeb9e02d6080daf3 in current term 2: Already voted for candidate 84759789d77f4c468c227d234125c57e in this term.
I20260806 06:51:11.285302 10755 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84759789d77f4c468c227d234125c57e in current term 2: Already voted for candidate 2b09e21849da4a27aeb9e02d6080daf3 in this term.
I20260806 06:51:11.285348 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.285138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41790)
0806 06:51:11.285299 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.285327 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260806 06:51:11.285454 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.285370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41792)
0806 06:51:11.285407 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.285434 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:51:11.285501 10732 leader_election.cc:334] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.285575 10732 leader_election.cc:302] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b09e21849da4a27aeb9e02d6080daf3; no voters: 675f3ce109c2436bb34fa69d23985b8c, 84759789d77f4c468c227d234125c57e
I20260806 06:51:11.285681 11452 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 06:51:11.285785 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.285649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:41798)
0806 06:51:11.285697 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.285754 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:51:11.285876 10918 leader_election.cc:334] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:11.285923 10918 leader_election.cc:302] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84759789d77f4c468c227d234125c57e; no voters: 2b09e21849da4a27aeb9e02d6080daf3, 675f3ce109c2436bb34fa69d23985b8c
I20260806 06:51:11.285996 11453 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 06:51:11.776398 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.776158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:57337)
0806 06:51:11.776301 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.776341 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260806 06:51:11.776451 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.776297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:38165)
0806 06:51:11.776386 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.776412 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":46,"threads_started":1}
I20260806 06:51:11.776398 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:11.776174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:39839)
0806 06:51:11.776302 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:11.776351 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 06:51:12.777066 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:12.776845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:37191)
0806 06:51:12.776967 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:12.777010 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260806 06:51:12.777114 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:12.776845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:36381)
0806 06:51:12.777006 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:12.777058 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
I20260806 06:51:12.777268 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:12.777032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:39205)
0806 06:51:12.777140 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:12.777171 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":45,"threads_started":1}
I20260806 06:51:13.170955 11468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:13.171049 11468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:13.171236 11468 leader_election.cc:288] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 84759789d77f4c468c227d234125c57e (127.3.247.188:35519)
I20260806 06:51:13.171502 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b09e21849da4a27aeb9e02d6080daf3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "84759789d77f4c468c227d234125c57e" is_pre_election: true
I20260806 06:51:13.171623 10941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b09e21849da4a27aeb9e02d6080daf3 in term 2.
I20260806 06:51:13.171638 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.171409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42060)
0806 06:51:13.171549 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.171592 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260806 06:51:13.171777 10731 leader_election.cc:302] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b09e21849da4a27aeb9e02d6080daf3, 84759789d77f4c468c227d234125c57e; no voters:
I20260806 06:51:13.171907 11468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 06:51:13.171958 11468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:13.171981 11468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:51:13.172097 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.171999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42072)
0806 06:51:13.172043 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.172069 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:51:13.172184 10732 leader_election.cc:334] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:13.172762 11468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:13.172900 11468 leader_election.cc:288] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 election: Requested vote from peers 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303), 84759789d77f4c468c227d234125c57e (127.3.247.188:35519)
I20260806 06:51:13.173101 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.172995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42076)
0806 06:51:13.173044 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.173074 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:51:13.173089 10941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b09e21849da4a27aeb9e02d6080daf3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "84759789d77f4c468c227d234125c57e"
I20260806 06:51:13.173187 10941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:51:13.173449 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.173330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42078)
0806 06:51:13.173401 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.173424 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 06:51:13.173560 10732 leader_election.cc:334] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
I20260806 06:51:13.173962 10941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b09e21849da4a27aeb9e02d6080daf3 in term 3.
I20260806 06:51:13.174132 10731 leader_election.cc:302] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b09e21849da4a27aeb9e02d6080daf3, 84759789d77f4c468c227d234125c57e; no voters: 675f3ce109c2436bb34fa69d23985b8c
I20260806 06:51:13.174216 11468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:51:13.174317 11468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [term 3 LEADER]: Becoming Leader. State: Replica: 2b09e21849da4a27aeb9e02d6080daf3, State: Running, Role: LEADER
I20260806 06:51:13.174398 11468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } }
I20260806 06:51:13.174769 11471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b09e21849da4a27aeb9e02d6080daf3. Latest consensus state: current_term: 3 leader_uuid: "2b09e21849da4a27aeb9e02d6080daf3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:13.174883 11471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:13.175088 11473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:13.175482 11473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:13.175540 11473 catalog_manager.cc:1259] Loaded cluster ID: 0f0abea884df4bd193d6b990f9fa8c5c
I20260806 06:51:13.175592 11473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:13.175789 11473 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:13.175853 11473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Loaded TSK: 0
I20260806 06:51:13.175999 11473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:13.198954 10745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:13.200614 10745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:13.201702 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.201577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42102)
0806 06:51:13.201639 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.201676 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:51:13.201820 10941 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [term 3 FOLLOWER]: Refusing update from remote peer 2b09e21849da4a27aeb9e02d6080daf3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 06:51:13.202112 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.202012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.0.0.1:42110)
0806 06:51:13.202064 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.202095 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:51:13.202068 11471 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 06:51:13.202205 10732 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 -> Peer 675f3ce109c2436bb34fa69d23985b8c (127.3.247.190:36303): Couldn't send request to peer 675f3ce109c2436bb34fa69d23985b8c. Status: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:51:13.202739 11475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b09e21849da4a27aeb9e02d6080daf3. Latest consensus state: current_term: 3 leader_uuid: "2b09e21849da4a27aeb9e02d6080daf3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:13.202847 11475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:51:13.203145 11474 mvcc.cc:205] Tried to move back new op lower bound from 7315452203832143872 to 7315452203723083776. Current Snapshot: MvccSnapshot[applied={T|T < 7315452203832143872}]
I20260806 06:51:13.203245 11471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2b09e21849da4a27aeb9e02d6080daf3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:13.203365 11471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:13.203562 11475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2b09e21849da4a27aeb9e02d6080daf3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "675f3ce109c2436bb34fa69d23985b8c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36303 } } peers { permanent_uuid: "2b09e21849da4a27aeb9e02d6080daf3" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 37323 } attrs { promote: false } } peers { permanent_uuid: "84759789d77f4c468c227d234125c57e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 35519 } attrs { promote: false } } }
I20260806 06:51:13.203655 11475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84759789d77f4c468c227d234125c57e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:51:13.208434 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.205221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44656 (local address 127.3.247.129:37119)
0806 06:51:13.205666 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.205669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.205676 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:13.205702 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.205702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.205712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.205779 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:13.205926 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206617 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.207674 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208086 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208175 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208187 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.208324 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.208369 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":50,"threads_started":1}
I20260806 06:51:13.208429 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.205088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37119 (local address 127.0.0.1:44656)
0806 06:51:13.205327 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.205336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:13.205384 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:13.205819 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:13.205819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:13.205821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:13.205854 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.205855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206701 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.206702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.207579 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.207581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208173 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.208232 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208236 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:13.208316 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:51:13.208357 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":4,"thread_start_us":114,"threads_started":1}
I20260806 06:51:13.208511 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.204989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:45657 (local address 127.0.0.1:40610)
0806 06:51:13.205034 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.205043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:13.205089 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:13.205441 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:13.205442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:13.205444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:13.205514 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.205515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206366 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.206367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.207516 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.207518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208302 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.208366 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208371 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:13.208401 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:51:13.208445 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:51:13.208738 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.205180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.3.247.131:45657)
0806 06:51:13.205294 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.205296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.205304 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:13.205332 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.205334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.205347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.205585 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:13.205630 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206265 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.207610 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208209 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208581 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208594 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.208612 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.208649 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260806 06:51:13.208846 11044 tablet_service.cc:1467] Processing CreateTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 (DEFAULT_TABLE table=test-table [id=69e2ed27adc64cfc8c384eb4c213cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:13.208982 11044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952d26c4f1f4274b68bf83d5b6d1ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:13.209012 11317 tablet_service.cc:1467] Processing CreateTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 (DEFAULT_TABLE table=test-table [id=69e2ed27adc64cfc8c384eb4c213cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:13.209136 11317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952d26c4f1f4274b68bf83d5b6d1ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:13.209154 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.205088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:36913 (local address 127.0.0.1:55198)
0806 06:51:13.205221 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.205231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:13.205345 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:13.205603 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:13.205604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:13.205606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:13.205639 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.205640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206576 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.206577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.207655 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.207656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208449 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.208981 (+ 531us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208986 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:51:13.209038 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:13.209077 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260806 06:51:13.209388 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.205307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.3.247.130:36913)
0806 06:51:13.205469 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.205471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.205479 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:13.205504 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.205504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.205515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.205588 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:13.205700 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.206494 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.207753 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.208350 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.208410 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.208422 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.209277 (+ 855us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.209326 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":52,"threads_started":1}
I20260806 06:51:13.209578 11181 tablet_service.cc:1467] Processing CreateTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 (DEFAULT_TABLE table=test-table [id=69e2ed27adc64cfc8c384eb4c213cea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:13.209685 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952d26c4f1f4274b68bf83d5b6d1ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:13.210471 11485 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: Bootstrap starting.
I20260806 06:51:13.210745 11486 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Bootstrap starting.
I20260806 06:51:13.210963 11485 tablet_bootstrap.cc:654] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:13.211122 11486 tablet_bootstrap.cc:654] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:13.211351 11487 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Bootstrap starting.
I20260806 06:51:13.211706 11487 tablet_bootstrap.cc:654] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:13.211800 11486 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: No bootstrap required, opened a new log
I20260806 06:51:13.211861 11486 ts_tablet_manager.cc:1397] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:13.211982 11486 raft_consensus.cc:348] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.212081 11486 raft_consensus.cc:374] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:13.212105 11486 raft_consensus.cc:729] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ca86b739b714c618525f500c7c91ac1, State: Initialized, Role: FOLLOWER
I20260806 06:51:13.212178 11486 consensus_queue.cc:260] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.212281 11485 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: No bootstrap required, opened a new log
I20260806 06:51:13.212329 11485 ts_tablet_manager.cc:1397] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:13.212371 11487 tablet_bootstrap.cc:492] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: No bootstrap required, opened a new log
I20260806 06:51:13.212401 11486 ts_tablet_manager.cc:1428] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:13.212411 11487 ts_tablet_manager.cc:1397] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:13.212461 11485 raft_consensus.cc:348] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.212544 11487 raft_consensus.cc:348] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.212615 11487 raft_consensus.cc:374] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:13.212639 11487 raft_consensus.cc:729] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495bed5977624d109524bc9811388206, State: Initialized, Role: FOLLOWER
I20260806 06:51:13.212659 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.212573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:46351)
0806 06:51:13.212609 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.212634 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:51:13.212863 11485 raft_consensus.cc:374] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:13.212913 11485 raft_consensus.cc:729] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49496af4abc44429a3874ca583ee3b62, State: Initialized, Role: FOLLOWER
I20260806 06:51:13.212896 11487 consensus_queue.cc:260] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.212977 11485 consensus_queue.cc:260] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.213152 11485 ts_tablet_manager.cc:1428] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:13.213256 11487 ts_tablet_manager.cc:1428] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:13.213435 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.213349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:54703)
0806 06:51:13.213384 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.213408 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:51:13.213560 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.213427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:35493)
0806 06:51:13.213507 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.213535 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 06:51:13.213826 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.213704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:58515)
0806 06:51:13.213748 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.213771 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:51:13.213919 11385 heartbeater.cc:503] Master 127.3.247.189:37323 was elected leader, sending a full tablet report...
I20260806 06:51:13.214684 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.214540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:55117)
0806 06:51:13.214623 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.214656 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 06:51:13.214718 11112 heartbeater.cc:503] Master 127.3.247.189:37323 was elected leader, sending a full tablet report...
I20260806 06:51:13.214733 11249 heartbeater.cc:503] Master 127.3.247.189:37323 was elected leader, sending a full tablet report...
I20260806 06:51:13.214845 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.214756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:57609)
0806 06:51:13.214791 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.214819 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:51:13.261646 11491 raft_consensus.cc:482] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:13.261740 11491 raft_consensus.cc:504] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.262161 11491 leader_election.cc:288] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119), 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:13.265173 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.262411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:49789 (local address 127.3.247.130:36913)
0806 06:51:13.262474 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.262476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.262485 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:13.262575 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.262576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.262586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.262645 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:13.262848 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.263460 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264453 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.264814 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264897 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.264906 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.265060 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.265095 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:13.265182 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.262342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:36913 (local address 127.3.247.131:49789)
0806 06:51:13.262430 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.262443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:13.262497 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:13.262678 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:13.262679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:13.262681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:13.262761 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.262763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.263557 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.263558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.264356 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.264358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.264926 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.264986 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.264989 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:13.265054 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:51:13.265095 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":26}
I20260806 06:51:13.265172 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.262410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:38881 (local address 127.3.247.129:37119)
0806 06:51:13.262456 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.262458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.262650 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:51:13.262681 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.262682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.262692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.262774 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:13.262922 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.263541 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264473 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.264821 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264902 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.264910 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.265067 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.265106 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:51:13.265209 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.262342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37119 (local address 127.3.247.131:38881)
0806 06:51:13.262535 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.262547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:13.262622 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:13.262784 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:13.262785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:13.262787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:13.262859 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.262861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.263623 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.263624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.264366 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:13.264368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.264900 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.264901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:13.264981 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.264984 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:13.265060 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:51:13.265099 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":22,"thread_start_us":107,"threads_started":1}
I20260806 06:51:13.265527 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952d26c4f1f4274b68bf83d5b6d1ef8" candidate_uuid: "6ca86b739b714c618525f500c7c91ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495bed5977624d109524bc9811388206" is_pre_election: true
I20260806 06:51:13.265635 11201 raft_consensus.cc:2454] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ca86b739b714c618525f500c7c91ac1 in term 0.
I20260806 06:51:13.265746 11064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952d26c4f1f4274b68bf83d5b6d1ef8" candidate_uuid: "6ca86b739b714c618525f500c7c91ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49496af4abc44429a3874ca583ee3b62" is_pre_election: true
I20260806 06:51:13.265870 11064 raft_consensus.cc:2454] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ca86b739b714c618525f500c7c91ac1 in term 0.
I20260806 06:51:13.265843 11271 leader_election.cc:302] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495bed5977624d109524bc9811388206, 6ca86b739b714c618525f500c7c91ac1; no voters:
I20260806 06:51:13.265982 11491 raft_consensus.cc:2781] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:13.266027 11491 raft_consensus.cc:482] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:13.266047 11491 raft_consensus.cc:3037] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:13.266858 11491 raft_consensus.cc:504] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.266993 11491 leader_election.cc:288] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [CANDIDATE]: Term 1 election: Requested vote from peers 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119), 495bed5977624d109524bc9811388206 (127.3.247.130:36913)
I20260806 06:51:13.267143 11064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952d26c4f1f4274b68bf83d5b6d1ef8" candidate_uuid: "6ca86b739b714c618525f500c7c91ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49496af4abc44429a3874ca583ee3b62"
I20260806 06:51:13.267177 11201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952d26c4f1f4274b68bf83d5b6d1ef8" candidate_uuid: "6ca86b739b714c618525f500c7c91ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "495bed5977624d109524bc9811388206"
I20260806 06:51:13.267253 11064 raft_consensus.cc:3037] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:13.267257 11201 raft_consensus.cc:3037] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:13.267836 11201 raft_consensus.cc:2454] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ca86b739b714c618525f500c7c91ac1 in term 1.
I20260806 06:51:13.268004 11271 leader_election.cc:302] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 495bed5977624d109524bc9811388206, 6ca86b739b714c618525f500c7c91ac1; no voters:
I20260806 06:51:13.268005 11064 raft_consensus.cc:2454] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ca86b739b714c618525f500c7c91ac1 in term 1.
I20260806 06:51:13.268112 11491 raft_consensus.cc:2781] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:13.268275 11491 raft_consensus.cc:686] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 1 LEADER]: Becoming Leader. State: Replica: 6ca86b739b714c618525f500c7c91ac1, State: Running, Role: LEADER
I20260806 06:51:13.268332 11491 consensus_queue.cc:237] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } }
I20260806 06:51:13.268616 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.268521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:55511)
0806 06:51:13.268551 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.268590 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:51:13.269214 10745 catalog_manager.cc:5495] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "6ca86b739b714c618525f500c7c91ac1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca86b739b714c618525f500c7c91ac1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 45657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260806 06:51:13.281955 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.279581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.3.247.131:45657)
0806 06:51:13.279643 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:13.279645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:13.279653 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:13.279689 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:13.279689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:13.279708 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:13.279796 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:13.279931 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.280413 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.281148 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:13.281362 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:13.281439 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:13.281594 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:13.281703 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:13.281790 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:13.281852 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:51:13.281873 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:51:13.282992 11201 raft_consensus.cc:1261] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Refusing update from remote peer 6ca86b739b714c618525f500c7c91ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:13.283119 11064 raft_consensus.cc:1261] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Refusing update from remote peer 6ca86b739b714c618525f500c7c91ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:13.283303 11495 consensus_queue.cc:1035] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495bed5977624d109524bc9811388206" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:13.283440 11491 consensus_queue.cc:1035] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49496af4abc44429a3874ca583ee3b62" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:13.284386 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.284269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:33703)
0806 06:51:13.284310 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.284353 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:51:13.284538 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.284370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:46123)
0806 06:51:13.284476 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.284507 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 06:51:13.285387 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.285232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:57873)
0806 06:51:13.285323 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.285354 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 06:51:13.285805 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.285671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.130:35413)
0806 06:51:13.285731 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.285756 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:51:13.286396 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.286317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.131:39063)
0806 06:51:13.286351 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.286372 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 06:51:13.287114 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:13.287022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36303 (local address 127.3.247.129:40041)
0806 06:51:13.287066 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.287092 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:51:13.288100 10745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:13.288184 10745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:13.289346 10745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b09e21849da4a27aeb9e02d6080daf3: Sending DeleteTablet for 3 replicas of tablet e952d26c4f1f4274b68bf83d5b6d1ef8
I20260806 06:51:13.289675 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:13 UTC) from {username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:55198
I20260806 06:51:13.289693 11044 tablet_service.cc:1514] Processing DeleteTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:13 UTC) from {username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:44656
I20260806 06:51:13.290069 11510 tablet_replica.cc:330] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62: stopping tablet replica
I20260806 06:51:13.290071 11317 tablet_service.cc:1514] Processing DeleteTablet for tablet e952d26c4f1f4274b68bf83d5b6d1ef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:13 UTC) from {username='oryx', principal='oryx/127.3.247.189@KRBTEST.COM'} at 127.0.0.1:40610
I20260806 06:51:13.290143 11510 raft_consensus.cc:2229] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:13.290192 11510 raft_consensus.cc:2258] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 49496af4abc44429a3874ca583ee3b62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:13.290282 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10892
I20260806 06:51:13.291218 11511 tablet_replica.cc:330] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: stopping tablet replica
I20260806 06:51:13.291270 11511 raft_consensus.cc:2229] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:13.291343 11511 raft_consensus.cc:2258] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:13.291620 11511 ts_tablet_manager.cc:1905] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:13.293363 11511 ts_tablet_manager.cc:1918] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:13.293426 11511 log.cc:1196] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/e952d26c4f1f4274b68bf83d5b6d1ef8
I20260806 06:51:13.293658 11511 ts_tablet_manager.cc:1939] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 6ca86b739b714c618525f500c7c91ac1: Deleting consensus metadata
I20260806 06:51:13.293876 11509 tablet_replica.cc:330] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: stopping tablet replica
I20260806 06:51:13.293987 10731 catalog_manager.cc:4841] TS 6ca86b739b714c618525f500c7c91ac1 (127.3.247.131:45657): tablet e952d26c4f1f4274b68bf83d5b6d1ef8 (table test-table [id=69e2ed27adc64cfc8c384eb4c213cea3]) successfully deleted
I20260806 06:51:13.294521 11509 raft_consensus.cc:2229] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:13.294588 11509 raft_consensus.cc:2258] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:13.294905 11509 ts_tablet_manager.cc:1905] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:13.296731 11509 ts_tablet_manager.cc:1918] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:13.296795 11509 log.cc:1196] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/e952d26c4f1f4274b68bf83d5b6d1ef8
I20260806 06:51:13.297034 11509 ts_tablet_manager.cc:1939] T e952d26c4f1f4274b68bf83d5b6d1ef8 P 495bed5977624d109524bc9811388206: Deleting consensus metadata
I20260806 06:51:13.297506 10730 catalog_manager.cc:4841] TS 495bed5977624d109524bc9811388206 (127.3.247.130:36913): tablet e952d26c4f1f4274b68bf83d5b6d1ef8 (table test-table [id=69e2ed27adc64cfc8c384eb4c213cea3]) successfully deleted
W20260806 06:51:13.298537 11480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:13.298330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37119 (local address 127.0.0.1:44672)
0806 06:51:13.298443 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:13.298470 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37119: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 06:51:13.298605 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11118
W20260806 06:51:13.298661 10729 catalog_manager.cc:4584] TS 49496af4abc44429a3874ca583ee3b62 (127.3.247.129:37119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e952d26c4f1f4274b68bf83d5b6d1ef8: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37119: connect: Connection reset by peer (error 104)
I20260806 06:51:13.303478 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11254
I20260806 06:51:13.309325 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10611
I20260806 06:51:13.314459 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 10799
2026-08-06T06:51:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6491 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:13 dist-test-slave-0kls krb5kdc[11514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:13 dist-test-slave-0kls krb5kdc[11514](info): set up 2 sockets
Aug 06 06:51:13 dist-test-slave-0kls krb5kdc[11514](info): commencing operation
krb5kdc: starting...
W20260806 06:51:14.362504 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab.
Entry for principal oryx/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab.
I20260806 06:51:14.481156 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33671
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab
--principal=oryx/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:14.561896 11527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:14.562115 11527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:14.562137 11527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:14.563261 11527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:14.563292 11527 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:51:14.563306 11527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:14.563319 11527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:14.563331 11527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:14.563344 11527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:14.564754 11527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33671
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.190.keytab
--principal=oryx/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:14.564942 11527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:14.565115 11527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:14.570006 11533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:14.570035 11527 server_base.cc:1034] running on GCE node
W20260806 06:51:14.569983 11532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:14.570005 11535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:14.570431 11527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:14.570710 11527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:14.571878 11527 hybrid_clock.cc:648] HybridClock initialized: now 1785999074571870 us; error 33 us; skew 500 ppm
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:14.574249 11527 init.cc:377] Logged in from keytab as oryx/127.3.247.190@KRBTEST.COM (short username oryx)
I20260806 06:51:14.574682 11527 webserver.cc:460] Webserver started at http://127.3.247.190:45737/ using document root <none> and password file <none>
I20260806 06:51:14.574914 11527 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:14.574982 11527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:14.575134 11527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:14.576319 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "68fe482ca30b467fa7f1aad634d038bc"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "e7603d876a0c0fe0040536867e462ace"
server_key_iv: "29d9aaedb733a2510851d6029277ae97"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.576694 11527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "68fe482ca30b467fa7f1aad634d038bc"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "e7603d876a0c0fe0040536867e462ace"
server_key_iv: "29d9aaedb733a2510851d6029277ae97"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.578058 11527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:14.578833 11542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.579139 11527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:14.579272 11527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "68fe482ca30b467fa7f1aad634d038bc"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "e7603d876a0c0fe0040536867e462ace"
server_key_iv: "29d9aaedb733a2510851d6029277ae97"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.579360 11527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:14.602687 11527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:14.604759 11527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:14.604945 11527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:14.609537 11527 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:33671
I20260806 06:51:14.609643 11604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:33671 every 8 connection(s)
I20260806 06:51:14.609972 11527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:14.610625 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:14.612880 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Bootstrap starting.
I20260806 06:51:14.613476 11605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:14.613798 11605 log.cc:824] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:14.614461 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: No bootstrap required, opened a new log
I20260806 06:51:14.615908 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11527
I20260806 06:51:14.616044 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:14.616246 4062 external_mini_cluster.cc:1428] Setting key cd4a17ad402625ca2e2f1cac546c00e4
I20260806 06:51:14.616788 11605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } }
I20260806 06:51:14.616979 11605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:14.617018 11605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68fe482ca30b467fa7f1aad634d038bc, State: Initialized, Role: FOLLOWER
I20260806 06:51:14.617143 11605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } }
I20260806 06:51:14.617221 11605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:14.617251 11605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:14.617287 11605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:14.617905 11605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } }
I20260806 06:51:14.617990 11605 leader_election.cc:302] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68fe482ca30b467fa7f1aad634d038bc; no voters:
I20260806 06:51:14.618157 11605 leader_election.cc:288] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:14.618189 11610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:14.618297 11610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [term 1 LEADER]: Becoming Leader. State: Replica: 68fe482ca30b467fa7f1aad634d038bc, State: Running, Role: LEADER
I20260806 06:51:14.618373 11610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } }
I20260806 06:51:14.618405 11605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:14.618923 11612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68fe482ca30b467fa7f1aad634d038bc. Latest consensus state: current_term: 1 leader_uuid: "68fe482ca30b467fa7f1aad634d038bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } } }
I20260806 06:51:14.619069 11612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:14.619647 11618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:14.620157 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68fe482ca30b467fa7f1aad634d038bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68fe482ca30b467fa7f1aad634d038bc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33671 } } }
I20260806 06:51:14.620216 11618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:14.620232 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:14.621989 11618 catalog_manager.cc:1347] Generated new cluster ID: 9047e879a836414ba8cf92d9037551ad
I20260806 06:51:14.622056 11618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:14.628346 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.616796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.3.247.190:33671)
0806 06:51:14.616992 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:14.617002 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:14.617045 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:51:14.617171 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:14.617178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:14.617236 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:14.617384 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:14.618085 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.619422 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.620647 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.621245 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.621351 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.624030 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:14.624034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:14.624034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:14.624047 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:14.626683 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.627061 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.627066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.627066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.627192 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.627427 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.627428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.627429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.627755 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:14.627918 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:14.627998 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:51:14.628148 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260806 06:51:14.633414 11618 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:14.634018 11618 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:14.643499 11618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Generated new TSK 0
I20260806 06:51:14.643676 11618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
I20260806 06:51:14.676635 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33671
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:14.773818 11631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:14.774014 11631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:14.774039 11631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:14.775242 11631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:14.775282 11631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:14.775302 11631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:51:14.775319 11631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:14.776798 11631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:14.777127 11631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:14.777443 11631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:14.782893 11639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:14.782905 11637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:14.782893 11636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:14.783094 11631 server_base.cc:1034] running on GCE node
I20260806 06:51:14.783331 11631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:14.783586 11631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:14.784752 11631 hybrid_clock.cc:648] HybridClock initialized: now 1785999074784740 us; error 27 us; skew 500 ppm
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:14.787350 11631 init.cc:377] Logged in from keytab as oryx/127.3.247.129@KRBTEST.COM (short username oryx)
I20260806 06:51:14.787773 11631 webserver.cc:460] Webserver started at http://127.3.247.129:39015/ using document root <none> and password file <none>
I20260806 06:51:14.788004 11631 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:14.788074 11631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:14.788230 11631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:14.789307 11631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "c59b1ee5c2954308860b6a42330a62a4"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "64b383a48e8f930088ce2f8b76d07793"
server_key_iv: "31fbd60f0f994d60e4da73e08aaadaa2"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.789673 11631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "c59b1ee5c2954308860b6a42330a62a4"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "64b383a48e8f930088ce2f8b76d07793"
server_key_iv: "31fbd60f0f994d60e4da73e08aaadaa2"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.791033 11631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:14.791855 11646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.792045 11631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:14.792160 11631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "c59b1ee5c2954308860b6a42330a62a4"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "64b383a48e8f930088ce2f8b76d07793"
server_key_iv: "31fbd60f0f994d60e4da73e08aaadaa2"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.792233 11631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:14.809931 11631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:14.811990 11631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:14.812156 11631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:14.812672 11631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:14.812753 11631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.812800 11631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:14.812831 11631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.820147 11631 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:35539
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:14.820273 11761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:35539 every 8 connection(s)
I20260806 06:51:14.820520 11631 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:14.822304 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11631
I20260806 06:51:14.822422 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:14.822657 4062 external_mini_cluster.cc:1428] Setting key 4e99a98ea4a5b92aa2e405a15cfa5db9
I20260806 06:51:14.823694 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.813230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.129:60165)
0806 06:51:14.813373 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:14.813409 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:14.814000 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:14.814262 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:14.814265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:14.814705 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:14.814997 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.815005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.816005 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.816007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.817490 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.817494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.818108 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.818110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.818256 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.818775 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:14.818804 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:14.820638 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:14.822155 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.822161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.822163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.822386 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.822630 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.822631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.822631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.822763 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.823146 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:14.823151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:14.823292 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:51:14.823408 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:51:14.823537 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260806 06:51:14.823696 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.813228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:60165 (local address 127.3.247.190:33671)
0806 06:51:14.813279 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:14.813281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:14.813514 (+ 233us) server_negotiation.cc:407] Connection header received
0806 06:51:14.814118 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:14.814119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:14.814132 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:14.814193 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:14.815087 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.815911 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.817595 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.817969 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.818084 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.820752 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:14.820754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:14.820754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:14.820762 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:14.822072 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.822488 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.822489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.822489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.822557 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.822925 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.822926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.822927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.823080 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:14.823149 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:14.823455 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 06:51:14.823551 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:51:14.824276 11762 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33671
I20260806 06:51:14.824366 11762 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:14.824576 11762 heartbeater.cc:511] Master 127.3.247.190:33671 requested a full tablet report, sending...
I20260806 06:51:14.825165 11559 ts_manager.cc:194] Registered new tserver with Master: c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539)
I20260806 06:51:14.825757 11559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:60165
WARNING: no policy specified for oryx/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:51:14.832244 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.826139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:43007 (local address 127.3.247.190:33671)
0806 06:51:14.826176 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:14.826178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:14.826714 (+ 536us) server_negotiation.cc:407] Connection header received
0806 06:51:14.826799 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:14.826800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:14.826812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:14.826882 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:14.827233 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.827704 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.828394 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.828606 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.828692 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.829635 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:14.829636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:14.829637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:14.829643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:14.831104 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.831449 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.831450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.831451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.831513 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.831796 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.831797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.831797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.831914 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:14.831976 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:14.832117 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 06:51:14.832183 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:51:14.832252 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.826083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.129:43007)
0806 06:51:14.826594 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:14.826605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:14.826665 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:14.826902 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:14.826902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:14.827089 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:14.827147 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.827148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.827801 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.827802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.828309 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.828310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.828693 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.828694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.828763 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.829220 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:14.829224 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:14.829520 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:14.831186 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.831188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.831188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.831361 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.831589 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.831641 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.831642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.831720 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.831982 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:14.831982 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:14.832041 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 06:51:14.832100 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:51:14.832178 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
I20260806 06:51:14.849109 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33671
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:14.930795 11771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:14.931001 11771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:14.931026 11771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:14.932206 11771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:14.932255 11771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:14.932276 11771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:51:14.932292 11771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:14.933903 11771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:14.934137 11771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:14.934381 11771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:14.939249 11776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:14.939242 11779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:14.939244 11777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:14.939494 11771 server_base.cc:1034] running on GCE node
I20260806 06:51:14.939669 11771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:14.939888 11771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:14.941023 11771 hybrid_clock.cc:648] HybridClock initialized: now 1785999074941012 us; error 27 us; skew 500 ppm
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:14.943466 11771 init.cc:377] Logged in from keytab as oryx/127.3.247.130@KRBTEST.COM (short username oryx)
I20260806 06:51:14.943837 11771 webserver.cc:460] Webserver started at http://127.3.247.130:35453/ using document root <none> and password file <none>
I20260806 06:51:14.944007 11771 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:14.944046 11771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:14.944132 11771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:14.945117 11771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "c2c4c91b941c468fb73f7706a7ae708d"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "155855146e095eaff39443b93f354d7e"
server_key_iv: "df11d3f47671de14f1bc70b5d05f2c36"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.945415 11771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "c2c4c91b941c468fb73f7706a7ae708d"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "155855146e095eaff39443b93f354d7e"
server_key_iv: "df11d3f47671de14f1bc70b5d05f2c36"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.946724 11771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:14.947383 11786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.947549 11771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:14.947630 11771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "c2c4c91b941c468fb73f7706a7ae708d"
format_stamp: "Formatted at 2026-08-06 06:51:14 on dist-test-slave-0kls"
server_key: "155855146e095eaff39443b93f354d7e"
server_key_iv: "df11d3f47671de14f1bc70b5d05f2c36"
server_key_version: "encryptionkey@0"
I20260806 06:51:14.947685 11771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:14.966017 11771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:14.967933 11771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:14.968043 11771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:14.968515 11771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:14.968554 11771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:14.968580 11771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:14.968592 11771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:51:14 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:14.975581 11771 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:39939
I20260806 06:51:14.975664 11901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:39939 every 8 connection(s)
I20260806 06:51:14.975980 11771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:14.978516 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.969037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:51229 (local address 127.3.247.190:33671)
0806 06:51:14.969121 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:14.969124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:14.969289 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:51:14.969778 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:14.969804 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:14.969819 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:14.969896 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:14.970750 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.971578 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.973234 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.973562 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.973657 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.975992 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:14.975993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:14.975994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:14.976001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:14.977387 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.977741 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.977742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.977742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.977832 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.978033 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.978033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.978034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.978143 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:14.978194 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:14.978399 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 06:51:14.978455 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:51:14.978749 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.968969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.130:51229)
0806 06:51:14.969139 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:14.969193 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:14.969676 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:14.969997 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:14.970001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:14.970354 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:14.970636 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.970643 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.971700 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.971701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.973131 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.973135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.973662 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.973662 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.973820 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.974344 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:14.974353 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:14.975895 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:14.977456 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.977461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.977463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.977675 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.977883 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.977883 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.977884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.977980 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.978199 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:14.978203 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:14.978309 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 06:51:14.978388 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:51:14.978498 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260806 06:51:14.979193 11902 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33671
I20260806 06:51:14.979290 11902 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:14.979457 11902 heartbeater.cc:511] Master 127.3.247.190:33671 requested a full tablet report, sending...
I20260806 06:51:14.979827 11558 ts_manager.cc:194] Registered new tserver with Master: c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939)
I20260806 06:51:14.980285 11558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:51229
I20260806 06:51:14.984097 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11771
I20260806 06:51:14.984222 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:14.984469 4062 external_mini_cluster.cc:1428] Setting key 3f727f3e44237485d9be6993151f6754
I20260806 06:51:14.985545 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.980165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.130:56817)
0806 06:51:14.980295 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:14.980307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:14.980368 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:14.980497 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:14.980498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:14.980661 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:14.980717 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.980718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.981340 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.981341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.981998 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:14.982000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.982364 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.982365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:14.982435 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.982773 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:14.982777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:14.983034 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:14.984440 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.984441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.984441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.984687 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.984927 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:14.984928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:14.984928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:14.984990 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:14.985309 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:14.985310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:14.985347 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 06:51:14.985404 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:51:14.985478 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260806 06:51:14.985638 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:14.980248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:56817 (local address 127.3.247.190:33671)
0806 06:51:14.980294 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:14.980296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:14.980354 (+ 58us) server_negotiation.cc:407] Connection header received
0806 06:51:14.980416 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:14.980417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:14.980426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:14.980491 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:14.980793 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.981247 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.982085 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:14.982279 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:14.982362 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:14.983130 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:14.983131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:14.983132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:14.983138 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:14.984364 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.984803 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.984804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.984805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.984851 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:14.985061 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:14.985062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:14.985062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:14.985235 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:14.985311 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:14.985444 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:51:14.985518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
WARNING: no policy specified for oryx/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
I20260806 06:51:15.008143 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:33671
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:15.087368 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:15.087559 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:15.087581 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:15.088618 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:15.088649 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:15.088667 11911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:51:15.088683 11911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:15.090107 11911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.247.190:33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:15.090289 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:15.090452 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:15.095378 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:15.095435 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:15.095460 11911 server_base.cc:1034] running on GCE node
W20260806 06:51:15.095403 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:15.095810 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:15.096055 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:15.097195 11911 hybrid_clock.cc:648] HybridClock initialized: now 1785999075097186 us; error 39 us; skew 500 ppm
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:15.099732 11911 init.cc:377] Logged in from keytab as oryx/127.3.247.131@KRBTEST.COM (short username oryx)
I20260806 06:51:15.100158 11911 webserver.cc:460] Webserver started at http://127.3.247.131:41543/ using document root <none> and password file <none>
I20260806 06:51:15.100376 11911 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:15.100419 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:15.100574 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:15.101612 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
format_stamp: "Formatted at 2026-08-06 06:51:15 on dist-test-slave-0kls"
server_key: "736e9388a92cee1bd4271bf941561471"
server_key_iv: "2cf3aab0db936a439010d938bcbbf1e3"
server_key_version: "encryptionkey@0"
I20260806 06:51:15.102013 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
format_stamp: "Formatted at 2026-08-06 06:51:15 on dist-test-slave-0kls"
server_key: "736e9388a92cee1bd4271bf941561471"
server_key_iv: "2cf3aab0db936a439010d938bcbbf1e3"
server_key_version: "encryptionkey@0"
I20260806 06:51:15.103292 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:51:15.103997 11926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.104192 11911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.104274 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
format_stamp: "Formatted at 2026-08-06 06:51:15 on dist-test-slave-0kls"
server_key: "736e9388a92cee1bd4271bf941561471"
server_key_iv: "2cf3aab0db936a439010d938bcbbf1e3"
server_key_version: "encryptionkey@0"
I20260806 06:51:15.104324 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:15.113981 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:15.115865 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:15.115999 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:15.116475 11911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:15.116514 11911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.116535 11911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:15.116549 11911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.123559 11911 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:39337
I20260806 06:51:15.123628 12041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:39337 every 8 connection(s)
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:15.123997 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:15.126688 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.116905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:48313 (local address 127.3.247.190:33671)
0806 06:51:15.116978 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.116981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.117190 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:51:15.117646 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.117648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.117661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.117746 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.118597 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.119479 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.121173 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.121501 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.121580 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.124202 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:15.124203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:15.124204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:15.124213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:15.125516 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.125937 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.125938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.125938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.125987 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.126213 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.126213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.126214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.126311 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:15.126359 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.126562 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.126622 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:51:15.126914 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.116879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.131:48313)
0806 06:51:15.117052 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.117090 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.117551 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.117865 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.117868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.118208 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:15.118490 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.118496 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.119642 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.119644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.121076 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.121079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.121593 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.121594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.121739 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.122344 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:15.122355 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:15.124095 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:15.125583 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.125589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.125590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.125870 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.126046 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.126047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.126047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.126149 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.126363 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:15.126367 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:15.126471 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.126543 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.126662 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260806 06:51:15.127373 12042 heartbeater.cc:347] Connected to a master server at 127.3.247.190:33671
I20260806 06:51:15.127450 12042 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:15.127630 12042 heartbeater.cc:511] Master 127.3.247.190:33671 requested a full tablet report, sending...
I20260806 06:51:15.128007 11558 ts_manager.cc:194] Registered new tserver with Master: b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:15.128504 11558 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:48313
I20260806 06:51:15.132714 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 11911
I20260806 06:51:15.132863 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:15.133105 4062 external_mini_cluster.cc:1428] Setting key 5944b9a28306c431fe0d31d36b7c3e5b
I20260806 06:51:15.133955 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.128375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:41539 (local address 127.3.247.190:33671)
0806 06:51:15.128637 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.128639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.128647 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:15.128666 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.128667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.128676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.128728 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.129073 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.129515 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.130223 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.130394 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.130447 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.131400 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:15.131401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:15.131402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:15.131407 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:15.132778 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.133164 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.133165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.133166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.133224 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.133429 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.133430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.133431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.133534 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:15.133604 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.133774 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.133868 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260806 06:51:15.133965 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.128309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.3.247.131:41539)
0806 06:51:15.128424 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.128436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.128489 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.128734 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.128736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.128931 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:15.128993 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.128995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.129599 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.129600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.130153 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.130155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.130495 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.130496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.130587 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.130990 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:15.130998 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:15.131316 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:15.132878 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.132880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.132880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.133073 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.133296 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.133297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.133298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.133356 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.133608 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:15.133609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:15.133653 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.133733 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.133856 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260806 06:51:15.134167 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:15.157924 11609 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:15.151939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.3.247.190:33671)
0806 06:51:15.152001 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.152002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.152057 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:51:15.152109 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.152110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.152118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.152171 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.152710 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.153205 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.154233 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.154412 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.154464 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.157771 (+ 3307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33982: BlockingRecv error: recv got EOF from 127.0.0.1:33982 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:15.164922 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.158734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.3.247.190:33671)
0806 06:51:15.158813 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.158815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.158853 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:51:15.158909 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.158909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.158917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.158968 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.159396 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.159853 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.160478 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.160668 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.160725 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.162819 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:15.162820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:15.162821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:15.162827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:15.164033 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.164353 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.164354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.164354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.164404 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.164563 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.164564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.164564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.164656 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:15.164729 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.164805 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.164859 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:51:15.166604 11558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:15.168130 11558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:15.174649 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:39939 (local address 127.0.0.1:48706)
0806 06:51:15.170621 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.170645 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.170699 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.171279 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.171280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.171281 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.171307 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.171308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.172557 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.172558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.173582 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.173584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174378 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.174442 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174446 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.174508 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.174548 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260806 06:51:15.174746 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.0.0.1:50248)
0806 06:51:15.170452 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.170501 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.170571 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.171028 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.171030 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.171033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.171099 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.171101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.172610 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.172611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.173616 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.173619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174517 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.174584 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174588 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.174648 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.174680 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":3,"thread_start_us":46,"threads_started":1}
I20260806 06:51:15.174752 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.3.247.129:35539)
0806 06:51:15.170883 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.170888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.170898 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:15.170940 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.170942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.170961 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.171056 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.171229 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.171967 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.173395 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174151 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174226 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174301 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.174600 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.174654 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260806 06:51:15.174832 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.3.247.131:39337)
0806 06:51:15.170833 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.170839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.170848 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:15.170892 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.170893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.170905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.171021 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.171358 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.172487 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.173716 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174448 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174518 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174584 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.174676 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.174726 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":79,"threads_started":1}
I20260806 06:51:15.174556 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:35539 (local address 127.0.0.1:56444)
0806 06:51:15.170717 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.170726 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.170765 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.171123 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.171124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.171126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.171172 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.171173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.172097 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.172098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.173304 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.173306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174238 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.174324 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174342 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.174407 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.174475 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":59,"threads_started":1}
I20260806 06:51:15.175336 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.170438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.3.247.130:39939)
0806 06:51:15.171144 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.171149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.171158 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:15.171194 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.171195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.171209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.171285 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.171351 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.172467 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.173664 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.174267 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.174369 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.174444 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.175174 (+ 730us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.175240 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":124,"threads_started":1}
I20260806 06:51:15.175746 11696 tablet_service.cc:1467] Processing CreateTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 (DEFAULT_TABLE table=test-table [id=a344605d74fc46758aa62135cab6627d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.175760 11976 tablet_service.cc:1467] Processing CreateTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 (DEFAULT_TABLE table=test-table [id=a344605d74fc46758aa62135cab6627d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.176093 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b42934b9ec1642fa9df7799d8fa49f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.176106 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b42934b9ec1642fa9df7799d8fa49f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.176060 11836 tablet_service.cc:1467] Processing CreateTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 (DEFAULT_TABLE table=test-table [id=a344605d74fc46758aa62135cab6627d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.176393 11836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b42934b9ec1642fa9df7799d8fa49f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.179396 12070 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Bootstrap starting.
I20260806 06:51:15.179935 12071 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Bootstrap starting.
I20260806 06:51:15.180261 12070 tablet_bootstrap.cc:654] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.180565 12071 tablet_bootstrap.cc:654] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.180641 12072 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Bootstrap starting.
I20260806 06:51:15.180771 12070 log.cc:824] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:15.180871 12071 log.cc:824] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:15.181231 12072 tablet_bootstrap.cc:654] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.181582 12070 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: No bootstrap required, opened a new log
I20260806 06:51:15.181599 12072 log.cc:824] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:15.181638 12070 ts_tablet_manager.cc:1397] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:15.182209 12071 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: No bootstrap required, opened a new log
I20260806 06:51:15.182276 12071 ts_tablet_manager.cc:1397] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:15.183677 12070 raft_consensus.cc:348] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.183856 12070 raft_consensus.cc:374] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.183892 12070 raft_consensus.cc:729] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.184019 12070 consensus_queue.cc:260] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.184312 11762 heartbeater.cc:503] Master 127.3.247.190:33671 was elected leader, sending a full tablet report...
I20260806 06:51:15.184316 12071 raft_consensus.cc:348] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.184469 12071 raft_consensus.cc:374] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.184510 12071 raft_consensus.cc:729] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.184563 12070 ts_tablet_manager.cc:1428] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:15.184630 12071 consensus_queue.cc:260] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.184844 12071 ts_tablet_manager.cc:1428] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:15.184930 12042 heartbeater.cc:503] Master 127.3.247.190:33671 was elected leader, sending a full tablet report...
I20260806 06:51:15.185470 12072 tablet_bootstrap.cc:492] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: No bootstrap required, opened a new log
I20260806 06:51:15.185537 12072 ts_tablet_manager.cc:1397] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260806 06:51:15.186996 12072 raft_consensus.cc:348] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.187111 12072 raft_consensus.cc:374] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.187132 12072 raft_consensus.cc:729] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.187280 12072 consensus_queue.cc:260] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.187577 12072 ts_tablet_manager.cc:1428] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 06:51:15.187577 11902 heartbeater.cc:503] Master 127.3.247.190:33671 was elected leader, sending a full tablet report...
I20260806 06:51:15.192257 12076 raft_consensus.cc:482] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:15.192353 12076 raft_consensus.cc:504] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.192699 12076 leader_election.cc:288] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337), c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939)
I20260806 06:51:15.195785 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.193053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:38349 (local address 127.3.247.131:39337)
0806 06:51:15.193090 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.193092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.193098 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:51:15.193117 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.193117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.193125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.193199 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.193345 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.193978 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195099 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.195445 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195498 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.195504 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.195679 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.195720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:15.195801 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.192834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.3.247.129:38349)
0806 06:51:15.192876 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.192887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.192923 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.193201 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.193202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.193203 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.193254 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.193256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.194046 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.194047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.195012 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.195015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.195511 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.195608 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.195612 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.195665 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.195707 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":78}
I20260806 06:51:15.195858 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.192917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:60001 (local address 127.3.247.130:39939)
0806 06:51:15.192956 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.192957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.193291 (+ 334us) server_negotiation.cc:407] Connection header received
0806 06:51:15.193354 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.193355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.193364 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.193409 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.193559 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.194097 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195030 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.195515 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195585 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.195601 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.195750 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.195785 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:51:15.196017 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.192835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:39939 (local address 127.3.247.129:60001)
0806 06:51:15.193217 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.193228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.193280 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.193420 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.193421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.193434 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.193504 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.193505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.194174 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.194175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.194966 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.194968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.195605 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.195606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.195687 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.195691 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.195746 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.195785 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":204,"thread_start_us":110,"threads_started":1}
I20260806 06:51:15.196085 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b42934b9ec1642fa9df7799d8fa49f58" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" is_pre_election: true
I20260806 06:51:15.196203 11996 raft_consensus.cc:2454] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:15.196385 11647 leader_election.cc:302] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:15.196429 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b42934b9ec1642fa9df7799d8fa49f58" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d" is_pre_election: true
I20260806 06:51:15.196545 11856 raft_consensus.cc:2454] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:15.196648 12076 raft_consensus.cc:2781] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:15.196719 12076 raft_consensus.cc:482] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:15.196761 12076 raft_consensus.cc:3037] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.197646 12076 raft_consensus.cc:504] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.197777 12076 leader_election.cc:288] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Requested vote from peers b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337), c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939)
I20260806 06:51:15.197916 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b42934b9ec1642fa9df7799d8fa49f58" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
I20260806 06:51:15.197949 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b42934b9ec1642fa9df7799d8fa49f58" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d"
I20260806 06:51:15.197990 11996 raft_consensus.cc:3037] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.198019 11856 raft_consensus.cc:3037] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.198639 11996 raft_consensus.cc:2454] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:15.198649 11856 raft_consensus.cc:2454] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:15.198803 11648 leader_election.cc:302] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2c4c91b941c468fb73f7706a7ae708d, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:15.198905 12076 raft_consensus.cc:2781] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:15.199044 12076 raft_consensus.cc:686] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Becoming Leader. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Running, Role: LEADER
I20260806 06:51:15.199148 12076 consensus_queue.cc:237] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } }
I20260806 06:51:15.199874 11559 catalog_manager.cc:5495] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "c59b1ee5c2954308860b6a42330a62a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } health_report { overall_health: HEALTHY } } }
I20260806 06:51:15.210333 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.208361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.3.247.129:35539)
0806 06:51:15.208400 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.208402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.208410 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:15.208469 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.208469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.208482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.208535 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.208646 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.209053 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.209705 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.209929 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.209976 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.210116 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.210194 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.210243 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.210275 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.210289 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:15.211848 11996 raft_consensus.cc:1261] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.211884 11856 raft_consensus.cc:1261] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.212162 12080 consensus_queue.cc:1035] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.212268 12076 consensus_queue.cc:1035] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.214233 12083 mvcc.cc:205] Tried to move back new op lower bound from 7315452212065312768 to 7315452212016218112. Current Snapshot: MvccSnapshot[applied={T|T < 7315452212065312768}]
I20260806 06:51:15.214859 12085 mvcc.cc:205] Tried to move back new op lower bound from 7315452212065312768 to 7315452212016218112. Current Snapshot: MvccSnapshot[applied={T|T < 7315452212065312768}]
I20260806 06:51:15.216753 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.216842 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.217533 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Sending DeleteTablet for 3 replicas of tablet b42934b9ec1642fa9df7799d8fa49f58
I20260806 06:51:15.217866 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:50248
I20260806 06:51:15.217885 11696 tablet_service.cc:1514] Processing DeleteTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56444
I20260806 06:51:15.218068 11836 tablet_service.cc:1514] Processing DeleteTablet for tablet b42934b9ec1642fa9df7799d8fa49f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:48706
I20260806 06:51:15.218259 12095 tablet_replica.cc:330] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: stopping tablet replica
I20260806 06:51:15.218349 12094 tablet_replica.cc:330] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: stopping tablet replica
I20260806 06:51:15.218432 12094 raft_consensus.cc:2229] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.218461 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:51:15.218534 12094 raft_consensus.cc:2258] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:51:15.218605 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:15.218366 12095 raft_consensus.cc:2229] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.219022 12094 ts_tablet_manager.cc:1905] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.219696 11696 tablet_service.cc:1467] Processing CreateTablet for tablet a32ef29349644cb18bd672e095074b61 (DEFAULT_TABLE table=test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.219709 11836 tablet_service.cc:1467] Processing CreateTablet for tablet a32ef29349644cb18bd672e095074b61 (DEFAULT_TABLE table=test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.219738 11976 tablet_service.cc:1467] Processing CreateTablet for tablet a32ef29349644cb18bd672e095074b61 (DEFAULT_TABLE table=test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.219825 11836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32ef29349644cb18bd672e095074b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.219811 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32ef29349644cb18bd672e095074b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.219834 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a32ef29349644cb18bd672e095074b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.221071 12071 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Bootstrap starting.
I20260806 06:51:15.221071 12070 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Bootstrap starting.
I20260806 06:51:15.221179 12072 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Bootstrap starting.
I20260806 06:51:15.221477 12070 tablet_bootstrap.cc:654] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.221477 12071 tablet_bootstrap.cc:654] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.221585 12072 tablet_bootstrap.cc:654] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.222314 12095 raft_consensus.cc:2258] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.222695 12095 ts_tablet_manager.cc:1905] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.223506 12094 ts_tablet_manager.cc:1918] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.223557 12071 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: No bootstrap required, opened a new log
I20260806 06:51:15.223661 12071 ts_tablet_manager.cc:1397] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:51:15.223687 12094 log.cc:1196] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/b42934b9ec1642fa9df7799d8fa49f58
I20260806 06:51:15.223809 12071 raft_consensus.cc:348] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.223901 12071 raft_consensus.cc:374] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.223930 12071 raft_consensus.cc:729] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.223974 12071 consensus_queue.cc:260] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.224066 12071 ts_tablet_manager.cc:1428] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.224261 12094 ts_tablet_manager.cc:1939] T b42934b9ec1642fa9df7799d8fa49f58 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting consensus metadata
I20260806 06:51:15.224371 12095 ts_tablet_manager.cc:1918] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.224443 12095 log.cc:1196] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/b42934b9ec1642fa9df7799d8fa49f58
I20260806 06:51:15.224704 12095 ts_tablet_manager.cc:1939] T b42934b9ec1642fa9df7799d8fa49f58 P c2c4c91b941c468fb73f7706a7ae708d: Deleting consensus metadata
I20260806 06:51:15.224817 11543 catalog_manager.cc:4841] TS b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337): tablet b42934b9ec1642fa9df7799d8fa49f58 (table test-table [id=a344605d74fc46758aa62135cab6627d]) successfully deleted
I20260806 06:51:15.224867 12072 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: No bootstrap required, opened a new log
I20260806 06:51:15.224908 12072 ts_tablet_manager.cc:1397] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 06:51:15.225051 12072 raft_consensus.cc:348] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.225126 12072 raft_consensus.cc:374] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.225152 12072 raft_consensus.cc:729] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.225179 11544 catalog_manager.cc:4841] TS c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939): tablet b42934b9ec1642fa9df7799d8fa49f58 (table test-table [id=a344605d74fc46758aa62135cab6627d]) successfully deleted
I20260806 06:51:15.225205 12072 consensus_queue.cc:260] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.225291 12072 ts_tablet_manager.cc:1428] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.225574 12093 tablet_replica.cc:330] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: stopping tablet replica
I20260806 06:51:15.225692 12093 raft_consensus.cc:2229] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:15.225844 12093 raft_consensus.cc:2258] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.226344 12070 tablet_bootstrap.cc:492] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: No bootstrap required, opened a new log
I20260806 06:51:15.226385 12093 ts_tablet_manager.cc:1905] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.226410 12070 ts_tablet_manager.cc:1397] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:51:15.226559 12070 raft_consensus.cc:348] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.226634 12070 raft_consensus.cc:374] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.226661 12070 raft_consensus.cc:729] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.226706 12070 consensus_queue.cc:260] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.226796 12070 ts_tablet_manager.cc:1428] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260806 06:51:15.226790 11903 tablet.cc:2369] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:15.227950 12093 ts_tablet_manager.cc:1918] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.228044 12093 log.cc:1196] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/b42934b9ec1642fa9df7799d8fa49f58
I20260806 06:51:15.228302 12093 ts_tablet_manager.cc:1939] T b42934b9ec1642fa9df7799d8fa49f58 P c59b1ee5c2954308860b6a42330a62a4: Deleting consensus metadata
I20260806 06:51:15.228663 11544 catalog_manager.cc:4841] TS c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539): tablet b42934b9ec1642fa9df7799d8fa49f58 (table test-table [id=a344605d74fc46758aa62135cab6627d]) successfully deleted
I20260806 06:51:15.275872 12078 raft_consensus.cc:482] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:15.275988 12078 raft_consensus.cc:504] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.276358 12078 leader_election.cc:288] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:15.279966 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.276529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.3.247.130:44671)
0806 06:51:15.276667 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.276680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.276732 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.276888 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.276889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.276890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.276961 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.276963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.277678 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.277679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.278938 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.278940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.279714 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.279796 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.279799 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.279856 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.279895 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 06:51:15.279999 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.276600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:44671 (local address 127.3.247.131:39337)
0806 06:51:15.276744 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.276746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.276757 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:15.276788 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.276789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.276803 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.276865 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.277049 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.277589 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279008 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.279632 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279710 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.279719 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.279872 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.279915 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 06:51:15.280066 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.276636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:44781 (local address 127.3.247.129:35539)
0806 06:51:15.276680 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.276682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.276802 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:51:15.276902 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.276903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.276914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.276941 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.277193 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.277915 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279179 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.279655 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279735 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.279744 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.279933 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.279978 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:51:15.280314 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.276532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:35539 (local address 127.3.247.130:44781)
0806 06:51:15.276736 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.276748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.276827 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.277030 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.277031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.277033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.277105 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.277107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.278016 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.278017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.279095 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.279097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.279769 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.279770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.279848 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.279852 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.280191 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.280232 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":54,"thread_start_us":61,"threads_started":1}
I20260806 06:51:15.280269 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32ef29349644cb18bd672e095074b61" candidate_uuid: "c2c4c91b941c468fb73f7706a7ae708d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" is_pre_election: true
I20260806 06:51:15.280362 11996 raft_consensus.cc:2454] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2c4c91b941c468fb73f7706a7ae708d in term 0.
I20260806 06:51:15.280546 11787 leader_election.cc:302] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c2c4c91b941c468fb73f7706a7ae708d; no voters:
I20260806 06:51:15.280568 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32ef29349644cb18bd672e095074b61" candidate_uuid: "c2c4c91b941c468fb73f7706a7ae708d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59b1ee5c2954308860b6a42330a62a4" is_pre_election: true
I20260806 06:51:15.280668 12078 raft_consensus.cc:2781] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:15.280694 12078 raft_consensus.cc:482] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:15.280687 11716 raft_consensus.cc:2454] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2c4c91b941c468fb73f7706a7ae708d in term 0.
I20260806 06:51:15.280721 12078 raft_consensus.cc:3037] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.281562 12078 raft_consensus.cc:504] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.281708 12078 leader_election.cc:288] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [CANDIDATE]: Term 1 election: Requested vote from peers c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:15.281889 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32ef29349644cb18bd672e095074b61" candidate_uuid: "c2c4c91b941c468fb73f7706a7ae708d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59b1ee5c2954308860b6a42330a62a4"
I20260806 06:51:15.281879 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a32ef29349644cb18bd672e095074b61" candidate_uuid: "c2c4c91b941c468fb73f7706a7ae708d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
I20260806 06:51:15.281951 11996 raft_consensus.cc:3037] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.281950 11716 raft_consensus.cc:3037] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.282708 11996 raft_consensus.cc:2454] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2c4c91b941c468fb73f7706a7ae708d in term 1.
I20260806 06:51:15.282708 11716 raft_consensus.cc:2454] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2c4c91b941c468fb73f7706a7ae708d in term 1.
I20260806 06:51:15.282881 11787 leader_election.cc:302] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c2c4c91b941c468fb73f7706a7ae708d; no voters:
I20260806 06:51:15.282958 12078 raft_consensus.cc:2781] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:15.283066 12078 raft_consensus.cc:686] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 1 LEADER]: Becoming Leader. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Running, Role: LEADER
I20260806 06:51:15.283133 12078 consensus_queue.cc:237] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.283777 11558 catalog_manager.cc:5495] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d reported cstate change: term changed from 0 to 1, leader changed from <none> to c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130). New cstate: current_term: 1 leader_uuid: "c2c4c91b941c468fb73f7706a7ae708d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
W20260806 06:51:15.321288 11763 tablet.cc:2369] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999074, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.190@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:15.330451 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.322502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33671 (local address 127.0.0.1:34006)
0806 06:51:15.322673 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.322692 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.322753 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.322907 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.322908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.323205 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:15.323266 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.323267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.324198 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.324199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.324737 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.324738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.325196 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.325196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.325239 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.325861 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:15.325865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:15.327484 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:15.329335 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.329342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.329343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.329549 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.329779 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:15.329779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:15.329780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:15.329892 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:15.330183 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:15.330188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:15.330227 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.330279 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.330376 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260806 06:51:15.330578 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.322577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.3.247.190:33671)
0806 06:51:15.322651 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.322653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.322751 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:51:15.322824 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.322825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.322837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.322894 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.323343 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.324119 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.324811 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.325106 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.325175 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.327567 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:15.327569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:15.327570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:15.327577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:15.329254 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.329632 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.329633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.329634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.329702 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.329960 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.329961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.329962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.330115 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:15.330187 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.330323 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.330441 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":39}
I20260806 06:51:15.332763 11558 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:34006:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:51:15.333012 11558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:15.334441 11696 tablet_service.cc:1467] Processing CreateTablet for tablet 08deab650c7d4e938733466693a1ba8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=580fe9aea7cf422ba6c8be819a4eeb4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:51:15.334458 11836 tablet_service.cc:1467] Processing CreateTablet for tablet 08deab650c7d4e938733466693a1ba8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=580fe9aea7cf422ba6c8be819a4eeb4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:51:15.334486 11976 tablet_service.cc:1467] Processing CreateTablet for tablet 08deab650c7d4e938733466693a1ba8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=580fe9aea7cf422ba6c8be819a4eeb4c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:51:15.334605 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08deab650c7d4e938733466693a1ba8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.334605 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08deab650c7d4e938733466693a1ba8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.334605 11836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08deab650c7d4e938733466693a1ba8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.336099 12070 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: Bootstrap starting.
I20260806 06:51:15.336148 12071 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: Bootstrap starting.
I20260806 06:51:15.336400 12070 tablet_bootstrap.cc:654] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.336581 12071 tablet_bootstrap.cc:654] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.336920 12072 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: Bootstrap starting.
I20260806 06:51:15.336959 12070 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: No bootstrap required, opened a new log
I20260806 06:51:15.337005 12070 ts_tablet_manager.cc:1397] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:15.337126 12070 raft_consensus.cc:348] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.337190 12070 raft_consensus.cc:374] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.337204 12070 raft_consensus.cc:729] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.337260 12070 consensus_queue.cc:260] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.337296 12072 tablet_bootstrap.cc:654] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.337352 12070 ts_tablet_manager.cc:1428] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.337371 12088 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.337452 12088 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.338073 12072 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: No bootstrap required, opened a new log
I20260806 06:51:15.338127 12072 ts_tablet_manager.cc:1397] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:51:15.338373 12072 raft_consensus.cc:348] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.338438 12072 raft_consensus.cc:374] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.338456 12072 raft_consensus.cc:729] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.338493 12072 consensus_queue.cc:260] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.338570 12072 ts_tablet_manager.cc:1428] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.338701 12078 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.338762 12078 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.339123 12071 tablet_bootstrap.cc:492] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: No bootstrap required, opened a new log
I20260806 06:51:15.339171 12071 ts_tablet_manager.cc:1397] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:51:15.339293 12071 raft_consensus.cc:348] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.339367 12071 raft_consensus.cc:374] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.339385 12071 raft_consensus.cc:729] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.339422 12071 consensus_queue.cc:260] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.339504 12071 ts_tablet_manager.cc:1428] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.339620 12077 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.339708 12077 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.366237 12077 raft_consensus.cc:482] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:15.366348 12077 raft_consensus.cc:504] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.366771 12077 leader_election.cc:288] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539), c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939)
I20260806 06:51:15.369655 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.366956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:35539 (local address 127.3.247.131:36877)
0806 06:51:15.367019 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.367032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.367076 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.367356 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.367357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.367359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.367415 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.367417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.368076 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.368077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.368834 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.368836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.369387 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.369388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.369467 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.369471 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.369535 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.369578 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:51:15.369664 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.367053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36877 (local address 127.3.247.129:35539)
0806 06:51:15.367195 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.367198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.367207 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:15.367236 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.367237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.367246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.367294 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.367494 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.367984 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.368938 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.369301 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.369390 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.369398 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.369554 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.369594 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 06:51:15.370004 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08deab650c7d4e938733466693a1ba8f" candidate_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59b1ee5c2954308860b6a42330a62a4" is_pre_election: true
I20260806 06:51:15.370110 11716 raft_consensus.cc:2454] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0667b8f60bb49669eb9a6154d2e9e8a in term 0.
I20260806 06:51:15.370287 11928 leader_election.cc:302] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:15.370434 12077 raft_consensus.cc:2781] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:15.370491 12077 raft_consensus.cc:482] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:15.370524 12077 raft_consensus.cc:3037] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.370604 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.367212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:47365 (local address 127.3.247.130:39939)
0806 06:51:15.367292 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.367294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.367452 (+ 158us) server_negotiation.cc:407] Connection header received
0806 06:51:15.367525 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.367526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.367537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.367622 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:15.367814 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.368287 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.369610 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.370224 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.370331 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.370339 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.370491 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.370529 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:51:15.370658 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.367090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:39939 (local address 127.3.247.131:47365)
0806 06:51:15.367353 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.367364 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.367457 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.367627 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.367628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.367629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:15.367725 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.367728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.368377 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.368378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.369499 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.369502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.370332 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.370333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.370414 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.370418 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.370483 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.370522 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":169,"threads_started":1}
I20260806 06:51:15.371318 12077 raft_consensus.cc:504] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.371445 12077 leader_election.cc:288] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [CANDIDATE]: Term 1 election: Requested vote from peers c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539), c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939)
I20260806 06:51:15.371665 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08deab650c7d4e938733466693a1ba8f" candidate_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59b1ee5c2954308860b6a42330a62a4"
I20260806 06:51:15.371754 11716 raft_consensus.cc:3037] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.371773 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08deab650c7d4e938733466693a1ba8f" candidate_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d" is_pre_election: true
I20260806 06:51:15.371801 11855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08deab650c7d4e938733466693a1ba8f" candidate_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d"
I20260806 06:51:15.371871 11856 raft_consensus.cc:2454] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0667b8f60bb49669eb9a6154d2e9e8a in term 0.
I20260806 06:51:15.372484 11716 raft_consensus.cc:2454] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0667b8f60bb49669eb9a6154d2e9e8a in term 1.
I20260806 06:51:15.372680 11928 leader_election.cc:302] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c59b1ee5c2954308860b6a42330a62a4; no voters: c2c4c91b941c468fb73f7706a7ae708d
I20260806 06:51:15.372799 12077 raft_consensus.cc:2781] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:15.372973 12077 raft_consensus.cc:686] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 LEADER]: Becoming Leader. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Running, Role: LEADER
I20260806 06:51:15.373056 12077 consensus_queue.cc:237] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.373406 12112 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: TxnStatusTablet state changed. Reason: New leader b0667b8f60bb49669eb9a6154d2e9e8a. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.373497 12112 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:51:15.373682 12114 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:51:15.373822 12114 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:51:15.373857 11557 catalog_manager.cc:5495] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a reported cstate change: term changed from 0 to 1, leader changed from <none> to b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131). New cstate: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } health_report { overall_health: HEALTHY } } }
W20260806 06:51:15.374528 12043 tablet.cc:2369] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:15.388847 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.386110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.0.0.1:50254)
0806 06:51:15.386234 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.386244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.386291 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.386452 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.386453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.386454 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:51:15.386500 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.386502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.387385 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.387386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.387934 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.387935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.388376 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.388377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.388421 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.388435 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.388711 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.388718 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.388754 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.388790 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 06:51:15.388862 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.386176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.3.247.131:39337)
0806 06:51:15.386227 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.386230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.386277 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:51:15.386351 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.386352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.386370 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.386445 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.386544 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.387310 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.388012 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.388296 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.388364 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.388530 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.388644 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.388721 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.388761 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.388781 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:51:15.389997 11856 raft_consensus.cc:3037] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.390141 11716 raft_consensus.cc:1261] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Refusing update from remote peer b0667b8f60bb49669eb9a6154d2e9e8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.390403 12077 consensus_queue.cc:1035] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.390983 11856 raft_consensus.cc:1261] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Refusing update from remote peer b0667b8f60bb49669eb9a6154d2e9e8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.391155 12088 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: TxnStatusTablet state changed. Reason: New leader b0667b8f60bb49669eb9a6154d2e9e8a. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.391251 12088 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.391539 12077 consensus_queue.cc:1035] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.391902 12084 mvcc.cc:205] Tried to move back new op lower bound from 7315452212795445248 to 7315452212728643584. Current Snapshot: MvccSnapshot[applied={T|T < 7315452212795445248}]
I20260806 06:51:15.392259 12078 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: TxnStatusTablet state changed. Reason: New leader b0667b8f60bb49669eb9a6154d2e9e8a. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.392346 12078 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.392305 12088 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.392377 12088 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c59b1ee5c2954308860b6a42330a62a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.392715 12077 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.392804 12077 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:51:15.393637 12077 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.393716 12077 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P b0667b8f60bb49669eb9a6154d2e9e8a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:51:15.394986 12078 tablet_replica.cc:439] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } } }
I20260806 06:51:15.395064 12078 tablet_replica.cc:442] T 08deab650c7d4e938733466693a1ba8f P c2c4c91b941c468fb73f7706a7ae708d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:51:15.396555 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.393911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.3.247.130:39939)
0806 06:51:15.393979 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.393981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.393989 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:15.394023 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.394023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.394055 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.394136 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.394270 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.394873 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.395722 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.395981 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.396039 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.396209 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.396314 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.396416 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.396458 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.396476 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:51:15.401449 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.398915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:37515 (local address 127.3.247.131:39337)
0806 06:51:15.398957 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.398963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.399033 (+ 70us) server_negotiation.cc:407] Connection header received
0806 06:51:15.399107 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.399107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.399115 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.399161 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.399300 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.399732 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.400862 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.401029 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.401075 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.401253 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.401278 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.401314 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.401388 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.401402 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:51:15.401526 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.398854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.3.247.130:37515)
0806 06:51:15.398979 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.398990 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.399048 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.399182 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.399183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.399184 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:51:15.399239 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.399240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.399800 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.399801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.400782 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.400784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.401094 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.401095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.401162 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.401177 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.401331 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.401333 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.401385 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.401445 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 06:51:15.403800 11716 raft_consensus.cc:1261] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Refusing update from remote peer c2c4c91b941c468fb73f7706a7ae708d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.403820 11996 raft_consensus.cc:1261] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Refusing update from remote peer c2c4c91b941c468fb73f7706a7ae708d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.404090 12078 consensus_queue.cc:1035] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.404261 12100 consensus_queue.cc:1035] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.412209 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.409992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:38693 (local address 127.3.247.130:39939)
0806 06:51:15.410054 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.410055 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.410061 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:51:15.410077 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.410077 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.410084 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.410154 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.410294 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.410752 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.411493 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.411788 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.411849 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.412028 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.412053 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.412091 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.412144 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.412160 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:51:15.412258 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.409807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:39939 (local address 127.3.247.131:38693)
0806 06:51:15.409869 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:15.409880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:15.409928 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:15.410157 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:15.410158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:15.410159 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:51:15.410218 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.410220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.410835 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.410836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.411408 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:15.411409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.411861 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.411862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:15.411939 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.411954 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.412095 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.412097 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:15.412143 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:51:15.412185 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:51:15.419174 11558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.419274 11558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.419872 11558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Sending DeleteTablet for 3 replicas of tablet a32ef29349644cb18bd672e095074b61
I20260806 06:51:15.420188 11696 tablet_service.cc:1514] Processing DeleteTablet for tablet a32ef29349644cb18bd672e095074b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56444
I20260806 06:51:15.420229 11836 tablet_service.cc:1514] Processing DeleteTablet for tablet a32ef29349644cb18bd672e095074b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:48706
I20260806 06:51:15.420308 12093 tablet_replica.cc:330] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: stopping tablet replica
I20260806 06:51:15.420315 12095 tablet_replica.cc:330] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: stopping tablet replica
I20260806 06:51:15.420353 12093 raft_consensus.cc:2229] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.420382 12093 raft_consensus.cc:2258] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.420375 12095 raft_consensus.cc:2229] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:15.420456 12095 raft_consensus.cc:2258] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.420852 12095 ts_tablet_manager.cc:1905] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.420966 12093 ts_tablet_manager.cc:1905] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.422003 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet a32ef29349644cb18bd672e095074b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:50248
I20260806 06:51:15.422103 12094 tablet_replica.cc:330] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: stopping tablet replica
I20260806 06:51:15.422164 12094 raft_consensus.cc:2229] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.422205 12094 raft_consensus.cc:2258] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.422461 12095 ts_tablet_manager.cc:1918] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:51:15.422520 12095 log.cc:1196] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/a32ef29349644cb18bd672e095074b61
I20260806 06:51:15.422537 12094 ts_tablet_manager.cc:1905] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.422731 12095 ts_tablet_manager.cc:1939] T a32ef29349644cb18bd672e095074b61 P c2c4c91b941c468fb73f7706a7ae708d: Deleting consensus metadata
I20260806 06:51:15.424206 12094 ts_tablet_manager.cc:1918] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:51:15.424276 12094 log.cc:1196] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/a32ef29349644cb18bd672e095074b61
I20260806 06:51:15.424479 12094 ts_tablet_manager.cc:1939] T a32ef29349644cb18bd672e095074b61 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting consensus metadata
I20260806 06:51:15.424788 11543 catalog_manager.cc:4841] TS b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337): tablet a32ef29349644cb18bd672e095074b61 (table test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]) successfully deleted
I20260806 06:51:15.424850 12093 ts_tablet_manager.cc:1918] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:51:15.424908 12093 log.cc:1196] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/a32ef29349644cb18bd672e095074b61
I20260806 06:51:15.425133 12093 ts_tablet_manager.cc:1939] T a32ef29349644cb18bd672e095074b61 P c59b1ee5c2954308860b6a42330a62a4: Deleting consensus metadata
I20260806 06:51:15.425427 11544 catalog_manager.cc:4841] TS c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539): tablet a32ef29349644cb18bd672e095074b61 (table test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]) successfully deleted
I20260806 06:51:15.425501 11544 catalog_manager.cc:4841] TS c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939): tablet a32ef29349644cb18bd672e095074b61 (table test-table [id=aabf366c3bb04e34ad3e9d3b9ccdc845]) successfully deleted
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:15.446332 11609 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:15.440298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.3.247.190:33671)
0806 06:51:15.440345 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.440347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.441021 (+ 674us) server_negotiation.cc:407] Connection header received
0806 06:51:15.441120 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.441121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.441147 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.441206 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.441799 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.442302 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.443039 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.443240 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.443319 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.446271 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34014: BlockingRecv error: recv got EOF from 127.0.0.1:34014 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 06:51:15 dist-test-slave-0kls krb5kdc[11514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM
I20260806 06:51:15.454401 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.447330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.3.247.190:33671)
0806 06:51:15.447427 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.447429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.447437 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:15.447483 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.447484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.447493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.447547 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:15.447994 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.448459 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.449131 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.449338 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.449421 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.451709 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:15.451711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:15.451711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:15.451719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:15.453135 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.453511 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.453512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.453513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.453570 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:15.453780 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:15.453798 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:15.453798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:15.453933 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:15.454029 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.454211 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.454291 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 06:51:15.455930 11558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:15.456126 11558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:15.457466 11976 tablet_service.cc:1467] Processing CreateTablet for tablet 20bed4b1b0a84413a85475709acc0540 (DEFAULT_TABLE table=test-table [id=11b9c543f49e4db5b21372337f7ad864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.457513 11696 tablet_service.cc:1467] Processing CreateTablet for tablet 20bed4b1b0a84413a85475709acc0540 (DEFAULT_TABLE table=test-table [id=11b9c543f49e4db5b21372337f7ad864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.457604 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20bed4b1b0a84413a85475709acc0540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.457612 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20bed4b1b0a84413a85475709acc0540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.457901 11836 tablet_service.cc:1467] Processing CreateTablet for tablet 20bed4b1b0a84413a85475709acc0540 (DEFAULT_TABLE table=test-table [id=11b9c543f49e4db5b21372337f7ad864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.458015 11836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20bed4b1b0a84413a85475709acc0540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.459017 12071 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Bootstrap starting.
I20260806 06:51:15.459385 12071 tablet_bootstrap.cc:654] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.459403 12072 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Bootstrap starting.
I20260806 06:51:15.459849 12072 tablet_bootstrap.cc:654] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.460327 12071 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: No bootstrap required, opened a new log
I20260806 06:51:15.460371 12071 ts_tablet_manager.cc:1397] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:15.460500 12071 raft_consensus.cc:348] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.460557 12071 raft_consensus.cc:374] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.460575 12071 raft_consensus.cc:729] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.460615 12071 consensus_queue.cc:260] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.460700 12071 ts_tablet_manager.cc:1428] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.461565 12070 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Bootstrap starting.
I20260806 06:51:15.461892 12070 tablet_bootstrap.cc:654] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.462245 12072 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: No bootstrap required, opened a new log
I20260806 06:51:15.462301 12072 ts_tablet_manager.cc:1397] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260806 06:51:15.462466 12072 raft_consensus.cc:348] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.462548 12072 raft_consensus.cc:374] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.462585 12072 raft_consensus.cc:729] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.462647 12070 tablet_bootstrap.cc:492] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: No bootstrap required, opened a new log
I20260806 06:51:15.462643 12072 consensus_queue.cc:260] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.462683 12070 ts_tablet_manager.cc:1397] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:51:15.462735 12072 ts_tablet_manager.cc:1428] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.462780 12070 raft_consensus.cc:348] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.462841 12070 raft_consensus.cc:374] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.462860 12070 raft_consensus.cc:729] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.462917 12070 consensus_queue.cc:260] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.463017 12070 ts_tablet_manager.cc:1428] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.766280 12088 raft_consensus.cc:482] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:15.766392 12088 raft_consensus.cc:504] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.766570 12088 leader_election.cc:288] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:15.766842 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20bed4b1b0a84413a85475709acc0540" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" is_pre_election: true
I20260806 06:51:15.766842 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20bed4b1b0a84413a85475709acc0540" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d" is_pre_election: true
I20260806 06:51:15.766955 11856 raft_consensus.cc:2454] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:15.766955 11996 raft_consensus.cc:2454] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:15.767143 11647 leader_election.cc:302] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:15.767231 12088 raft_consensus.cc:2781] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:15.767271 12088 raft_consensus.cc:482] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:15.767295 12088 raft_consensus.cc:3037] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.768054 12088 raft_consensus.cc:504] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.768187 12088 leader_election.cc:288] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Requested vote from peers c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:15.768353 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20bed4b1b0a84413a85475709acc0540" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d"
I20260806 06:51:15.768389 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20bed4b1b0a84413a85475709acc0540" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
I20260806 06:51:15.768453 11856 raft_consensus.cc:3037] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.768460 11996 raft_consensus.cc:3037] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:15.769210 11856 raft_consensus.cc:2454] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:15.769210 11996 raft_consensus.cc:2454] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:15.769414 11648 leader_election.cc:302] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2c4c91b941c468fb73f7706a7ae708d, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:15.769519 12088 raft_consensus.cc:2781] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:15.769658 12088 raft_consensus.cc:686] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Becoming Leader. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Running, Role: LEADER
I20260806 06:51:15.769717 12088 consensus_queue.cc:237] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.770308 11558 catalog_manager.cc:5495] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "c59b1ee5c2954308860b6a42330a62a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260806 06:51:15.816124 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:15.813924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.3.247.129:35539)
0806 06:51:15.813972 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:15.813976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:15.813994 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:51:15.814098 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:15.814099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:15.814109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:15.814158 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:15.814281 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.814756 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.815525 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:15.815714 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:15.815764 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:15.815888 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:15.815913 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:15.815952 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:15.816045 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:51:15.816062 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:15.817101 11856 raft_consensus.cc:1261] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.817101 11996 raft_consensus.cc:1261] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:15.817351 12143 consensus_queue.cc:1035] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.817524 12088 consensus_queue.cc:1035] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:15.819682 11557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.819784 11557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:15.820480 11557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Sending DeleteTablet for 3 replicas of tablet 20bed4b1b0a84413a85475709acc0540
I20260806 06:51:15.820700 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet 20bed4b1b0a84413a85475709acc0540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:50248
I20260806 06:51:15.820727 11696 tablet_service.cc:1514] Processing DeleteTablet for tablet 20bed4b1b0a84413a85475709acc0540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56444
I20260806 06:51:15.820779 11836 tablet_service.cc:1514] Processing DeleteTablet for tablet 20bed4b1b0a84413a85475709acc0540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:15 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:48706
I20260806 06:51:15.820880 12095 tablet_replica.cc:330] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: stopping tablet replica
I20260806 06:51:15.820883 12094 tablet_replica.cc:330] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: stopping tablet replica
I20260806 06:51:15.820933 12095 raft_consensus.cc:2229] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.820935 12094 raft_consensus.cc:2229] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:15.820964 12095 raft_consensus.cc:2258] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.820979 12094 raft_consensus.cc:2258] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.821063 12093 tablet_replica.cc:330] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: stopping tablet replica
I20260806 06:51:15.821117 12093 raft_consensus.cc:2229] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:15.821165 12093 raft_consensus.cc:2258] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:15.821297 12095 ts_tablet_manager.cc:1905] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.821509 12094 ts_tablet_manager.cc:1905] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.822850 12095 ts_tablet_manager.cc:1918] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.822911 12095 log.cc:1196] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/20bed4b1b0a84413a85475709acc0540
I20260806 06:51:15.822870 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:15.823225 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:15.823272 12095 ts_tablet_manager.cc:1939] T 20bed4b1b0a84413a85475709acc0540 P c2c4c91b941c468fb73f7706a7ae708d: Deleting consensus metadata
I20260806 06:51:15.823263 12094 ts_tablet_manager.cc:1918] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.823366 12094 log.cc:1196] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/20bed4b1b0a84413a85475709acc0540
I20260806 06:51:15.823673 11544 catalog_manager.cc:4841] TS c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939): tablet 20bed4b1b0a84413a85475709acc0540 (table test-table [id=11b9c543f49e4db5b21372337f7ad864]) successfully deleted
I20260806 06:51:15.824071 12094 ts_tablet_manager.cc:1939] T 20bed4b1b0a84413a85475709acc0540 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting consensus metadata
I20260806 06:51:15.824381 11543 catalog_manager.cc:4841] TS b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337): tablet 20bed4b1b0a84413a85475709acc0540 (table test-table [id=11b9c543f49e4db5b21372337f7ad864]) successfully deleted
I20260806 06:51:15.824783 11976 tablet_service.cc:1467] Processing CreateTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 (DEFAULT_TABLE table=test-table [id=a2f26fea816f49d497dbb968610e1731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.824792 11696 tablet_service.cc:1467] Processing CreateTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 (DEFAULT_TABLE table=test-table [id=a2f26fea816f49d497dbb968610e1731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.824894 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d087a54df9dc4c4589f7d79a0b280b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.824895 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d087a54df9dc4c4589f7d79a0b280b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.825729 12093 ts_tablet_manager.cc:1905] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:15.825997 12071 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Bootstrap starting.
I20260806 06:51:15.826282 12070 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: Bootstrap starting.
I20260806 06:51:15.826332 12071 tablet_bootstrap.cc:654] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.826545 12070 tablet_bootstrap.cc:654] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.826932 12071 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: No bootstrap required, opened a new log
I20260806 06:51:15.826956 11836 tablet_service.cc:1467] Processing CreateTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 (DEFAULT_TABLE table=test-table [id=a2f26fea816f49d497dbb968610e1731]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:15.826990 12071 ts_tablet_manager.cc:1397] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:15.827059 11836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d087a54df9dc4c4589f7d79a0b280b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:15.827098 12071 raft_consensus.cc:348] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.827172 12071 raft_consensus.cc:374] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.827199 12071 raft_consensus.cc:729] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0667b8f60bb49669eb9a6154d2e9e8a, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.827225 12093 ts_tablet_manager.cc:1918] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:15.827280 12093 log.cc:1196] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/20bed4b1b0a84413a85475709acc0540
I20260806 06:51:15.827245 12071 consensus_queue.cc:260] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.827358 12071 ts_tablet_manager.cc:1428] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:15.827500 12093 ts_tablet_manager.cc:1939] T 20bed4b1b0a84413a85475709acc0540 P c59b1ee5c2954308860b6a42330a62a4: Deleting consensus metadata
I20260806 06:51:15.827876 12070 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: No bootstrap required, opened a new log
I20260806 06:51:15.827939 12070 ts_tablet_manager.cc:1397] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:51:15.828037 11544 catalog_manager.cc:4841] TS c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539): tablet 20bed4b1b0a84413a85475709acc0540 (table test-table [id=11b9c543f49e4db5b21372337f7ad864]) successfully deleted
I20260806 06:51:15.828045 12070 raft_consensus.cc:348] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.828107 12070 raft_consensus.cc:374] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.828127 12070 raft_consensus.cc:729] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.828169 12070 consensus_queue.cc:260] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.828258 12070 ts_tablet_manager.cc:1428] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:15.828423 12072 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Bootstrap starting.
I20260806 06:51:15.828776 12072 tablet_bootstrap.cc:654] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:15.829363 12072 tablet_bootstrap.cc:492] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: No bootstrap required, opened a new log
I20260806 06:51:15.829404 12072 ts_tablet_manager.cc:1397] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:51:15.829521 12072 raft_consensus.cc:348] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.829591 12072 raft_consensus.cc:374] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:15.829604 12072 raft_consensus.cc:729] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c4c91b941c468fb73f7706a7ae708d, State: Initialized, Role: FOLLOWER
I20260806 06:51:15.829674 12072 consensus_queue.cc:260] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:15.829768 12072 ts_tablet_manager.cc:1428] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:16.042001 12148 raft_consensus.cc:482] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:16.042104 12148 raft_consensus.cc:504] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:16.042272 12148 leader_election.cc:288] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:16.042518 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d087a54df9dc4c4589f7d79a0b280b76" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" is_pre_election: true
I20260806 06:51:16.042614 11996 raft_consensus.cc:2454] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:16.042531 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d087a54df9dc4c4589f7d79a0b280b76" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d" is_pre_election: true
I20260806 06:51:16.042655 11856 raft_consensus.cc:2454] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 0.
I20260806 06:51:16.042781 11647 leader_election.cc:302] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0667b8f60bb49669eb9a6154d2e9e8a, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:16.042893 12148 raft_consensus.cc:2781] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:16.042963 12148 raft_consensus.cc:482] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:16.042994 12148 raft_consensus.cc:3037] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:16.043701 12148 raft_consensus.cc:504] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:16.043848 12148 leader_election.cc:288] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Requested vote from peers c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939), b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337)
I20260806 06:51:16.044065 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d087a54df9dc4c4589f7d79a0b280b76" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c4c91b941c468fb73f7706a7ae708d"
I20260806 06:51:16.044065 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d087a54df9dc4c4589f7d79a0b280b76" candidate_uuid: "c59b1ee5c2954308860b6a42330a62a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a"
I20260806 06:51:16.044152 11996 raft_consensus.cc:3037] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:16.044150 11856 raft_consensus.cc:3037] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:16.044890 11856 raft_consensus.cc:2454] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:16.044890 11996 raft_consensus.cc:2454] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59b1ee5c2954308860b6a42330a62a4 in term 1.
I20260806 06:51:16.045085 11648 leader_election.cc:302] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2c4c91b941c468fb73f7706a7ae708d, c59b1ee5c2954308860b6a42330a62a4; no voters:
I20260806 06:51:16.045195 12148 raft_consensus.cc:2781] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:16.045289 12148 raft_consensus.cc:686] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Becoming Leader. State: Replica: c59b1ee5c2954308860b6a42330a62a4, State: Running, Role: LEADER
I20260806 06:51:16.045352 12148 consensus_queue.cc:237] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } }
I20260806 06:51:16.046097 11557 catalog_manager.cc:5495] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "c59b1ee5c2954308860b6a42330a62a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59b1ee5c2954308860b6a42330a62a4" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 35539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260806 06:51:16.058921 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:16.056691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:42977 (local address 127.3.247.131:39337)
0806 06:51:16.056805 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:16.056807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:16.056817 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:16.056870 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:16.056870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:16.056880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:16.056936 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:16.057042 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.057516 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.058223 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.058420 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.058484 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:16.058670 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:16.058697 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:16.058736 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:16.058844 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:51:16.058860 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260806 06:51:16.058950 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:16.056612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39337 (local address 127.3.247.129:42977)
0806 06:51:16.056765 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:16.056776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:16.056828 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:16.056953 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:16.056953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:16.056955 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:51:16.057001 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:16.057002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.057582 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.057582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:16.058158 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:16.058160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.058529 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.058530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:16.058579 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:16.058593 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:51:16.058783 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:51:16.058786 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:16.058840 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:51:16.058886 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260806 06:51:16.060989 11996 raft_consensus.cc:1261] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:16.060995 11856 raft_consensus.cc:1261] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Refusing update from remote peer c59b1ee5c2954308860b6a42330a62a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:16.061231 12143 consensus_queue.cc:1035] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c4c91b941c468fb73f7706a7ae708d" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:16.061354 12148 consensus_queue.cc:1035] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0667b8f60bb49669eb9a6154d2e9e8a" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:16.067870 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:16.065515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:44387 (local address 127.3.247.129:35539)
0806 06:51:16.065566 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:16.065568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:16.065676 (+ 108us) server_negotiation.cc:407] Connection header received
0806 06:51:16.065750 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:16.065751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:16.065762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:16.065859 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:16.065952 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.066403 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.067158 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.067432 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.067510 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:16.067639 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:16.067679 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:16.067730 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:16.067798 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:51:16.067816 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:51:16.067902 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:16.065486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:35539 (local address 127.3.247.131:44387)
0806 06:51:16.065626 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:16.065637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:16.065689 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:16.065872 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:16.065873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:16.065874 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:51:16.065914 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:16.065915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.066474 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.066476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:16.067071 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:16.067073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:16.067508 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:16.067509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:16.067571 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:16.067575 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:51:16.067745 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:51:16.067747 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:16.067790 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:51:16.067845 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260806 06:51:16.073300 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:16.073411 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:16.073999 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68fe482ca30b467fa7f1aad634d038bc: Sending DeleteTablet for 3 replicas of tablet d087a54df9dc4c4589f7d79a0b280b76
I20260806 06:51:16.074268 11696 tablet_service.cc:1514] Processing DeleteTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:16 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56444
I20260806 06:51:16.074297 11836 tablet_service.cc:1514] Processing DeleteTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:16 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:48706
I20260806 06:51:16.074368 12093 tablet_replica.cc:330] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4: stopping tablet replica
I20260806 06:51:16.074388 12095 tablet_replica.cc:330] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: stopping tablet replica
I20260806 06:51:16.074416 12093 raft_consensus.cc:2229] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:16.074427 12095 raft_consensus.cc:2229] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:16.074458 12095 raft_consensus.cc:2258] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:16.074491 12093 raft_consensus.cc:2258] T d087a54df9dc4c4589f7d79a0b280b76 P c59b1ee5c2954308860b6a42330a62a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:16.074589 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet d087a54df9dc4c4589f7d79a0b280b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:16 UTC) from {username='oryx', principal='oryx/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:50248
I20260806 06:51:16.074668 12094 tablet_replica.cc:330] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: stopping tablet replica
I20260806 06:51:16.074651 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11631
I20260806 06:51:16.074715 12094 raft_consensus.cc:2229] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:16.074750 12094 raft_consensus.cc:2258] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:16.076242 12095 ts_tablet_manager.cc:1905] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:16.076246 12094 ts_tablet_manager.cc:1905] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:16.077891 12094 ts_tablet_manager.cc:1918] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:51:16.077891 12095 ts_tablet_manager.cc:1918] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:51:16.077953 12094 log.cc:1196] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/d087a54df9dc4c4589f7d79a0b280b76
I20260806 06:51:16.077953 12095 log.cc:1196] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/d087a54df9dc4c4589f7d79a0b280b76
I20260806 06:51:16.078176 12094 ts_tablet_manager.cc:1939] T d087a54df9dc4c4589f7d79a0b280b76 P b0667b8f60bb49669eb9a6154d2e9e8a: Deleting consensus metadata
I20260806 06:51:16.078186 12095 ts_tablet_manager.cc:1939] T d087a54df9dc4c4589f7d79a0b280b76 P c2c4c91b941c468fb73f7706a7ae708d: Deleting consensus metadata
I20260806 06:51:16.078490 11544 catalog_manager.cc:4841] TS c2c4c91b941c468fb73f7706a7ae708d (127.3.247.130:39939): tablet d087a54df9dc4c4589f7d79a0b280b76 (table test-table [id=a2f26fea816f49d497dbb968610e1731]) successfully deleted
I20260806 06:51:16.078501 11543 catalog_manager.cc:4841] TS b0667b8f60bb49669eb9a6154d2e9e8a (127.3.247.131:39337): tablet d087a54df9dc4c4589f7d79a0b280b76 (table test-table [id=a2f26fea816f49d497dbb968610e1731]) successfully deleted
W20260806 06:51:16.081413 11544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:35539 (error 108)
I20260806 06:51:16.081651 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11771
I20260806 06:51:16.082754 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:16.081849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:35539 (local address 127.0.0.1:56470)
0806 06:51:16.082674 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:16.082718 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":42,"threads_started":1}
W20260806 06:51:16.082906 11544 catalog_manager.cc:4584] TS c59b1ee5c2954308860b6a42330a62a4 (127.3.247.129:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d087a54df9dc4c4589f7d79a0b280b76: Network error: Client connection negotiation failed: client connection to 127.3.247.129:35539: connect: Connection refused (error 111)
I20260806 06:51:16.088199 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11911
I20260806 06:51:16.093887 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 11527
2026-08-06T06:51:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2778 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:16 dist-test-slave-0kls krb5kdc[12168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:16 dist-test-slave-0kls krb5kdc[12168](info): set up 2 sockets
Aug 06 06:51:16 dist-test-slave-0kls krb5kdc[12168](info): commencing operation
krb5kdc: starting...
W20260806 06:51:17.138360 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:17 dist-test-slave-0kls krb5kdc[12168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:51:17.255941 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34021
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:34021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:17.337558 12181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:17.337769 12181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:17.337846 12181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:17.339032 12181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:17.339073 12181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:17.339087 12181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:17.339100 12181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:17.339114 12181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:17.340704 12181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:34021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:17.340950 12181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:17.341167 12181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:17.345957 12187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:17.346051 12181 server_base.cc:1034] running on GCE node
W20260806 06:51:17.345960 12186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:17.345971 12189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:17.346411 12181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:17.346705 12181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:17.347880 12181 hybrid_clock.cc:648] HybridClock initialized: now 1785999077347874 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T06:51:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1258 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:17 dist-test-slave-0kls krb5kdc[12194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:17 dist-test-slave-0kls krb5kdc[12194](info): set up 2 sockets
Aug 06 06:51:17 dist-test-slave-0kls krb5kdc[12194](info): commencing operation
krb5kdc: starting...
W20260806 06:51:18.390437 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:51:18.508287 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43507
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:43507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:18.590691 12207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:18.590901 12207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:18.590924 12207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:18.592028 12207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:18.592064 12207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:18.592078 12207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:18.592092 12207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:18.592103 12207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:18.593477 12207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:43507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:18.593662 12207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:18.593852 12207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:18.598690 12212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.598657 12215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.598661 12213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:18.598948 12207 server_base.cc:1034] running on GCE node
I20260806 06:51:18.599200 12207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:18.599485 12207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:18.600660 12207 hybrid_clock.cc:648] HybridClock initialized: now 1785999078600643 us; error 58 us; skew 500 ppm
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:18.607679 12207 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:51:18.608136 12207 webserver.cc:460] Webserver started at http://127.3.247.190:37713/ using document root <none> and password file <none>
I20260806 06:51:18.608378 12207 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:18.608448 12207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:18.608596 12207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:18.609666 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "9b92d0c380094ad295d8a2a0e0af2a69"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "bbd80af0f992596c4017186a269110e0"
server_key_iv: "6fca308d47381caee2bd780a1e62b7f4"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.610100 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "9b92d0c380094ad295d8a2a0e0af2a69"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "bbd80af0f992596c4017186a269110e0"
server_key_iv: "6fca308d47381caee2bd780a1e62b7f4"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.611442 12207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:18.612150 12222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.612323 12207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:18.612423 12207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "9b92d0c380094ad295d8a2a0e0af2a69"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "bbd80af0f992596c4017186a269110e0"
server_key_iv: "6fca308d47381caee2bd780a1e62b7f4"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.612509 12207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:18.627930 12207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:18.629776 12207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:18.630023 12207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:18.633956 12207 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:43507
I20260806 06:51:18.634001 12274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:43507 every 8 connection(s)
I20260806 06:51:18.634341 12207 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:18.635169 12275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:18.637321 12275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Bootstrap starting.
I20260806 06:51:18.637912 12275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:18.638152 12275 log.cc:824] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:18.638778 12275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: No bootstrap required, opened a new log
I20260806 06:51:18.640280 12275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:18.640411 12275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:18.640434 12275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b92d0c380094ad295d8a2a0e0af2a69, State: Initialized, Role: FOLLOWER
I20260806 06:51:18.640532 12275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:18.640623 12275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:18.640659 12275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:18.640710 12275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:18.641214 12275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:18.641333 12275 leader_election.cc:302] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b92d0c380094ad295d8a2a0e0af2a69; no voters:
I20260806 06:51:18.641502 12275 leader_election.cc:288] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:18.641551 12278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:18.641737 12278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 LEADER]: Becoming Leader. State: Replica: 9b92d0c380094ad295d8a2a0e0af2a69, State: Running, Role: LEADER
I20260806 06:51:18.641758 12275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:18.641858 12278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:18.642292 12279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } } }
I20260806 06:51:18.642319 12280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b92d0c380094ad295d8a2a0e0af2a69. Latest consensus state: current_term: 1 leader_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } } }
I20260806 06:51:18.642427 12279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:18.642481 12280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:18.642863 12284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:18.643149 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12207
I20260806 06:51:18.643256 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:18.643409 12284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:18.643455 4062 external_mini_cluster.cc:1428] Setting key 91f220dad3b873466a3d32400cbb3aca
I20260806 06:51:18.645057 12284 catalog_manager.cc:1347] Generated new cluster ID: 762e859d24174ed0b7629c6bb7a399ea
I20260806 06:51:18.645143 12284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:18.656915 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:18.643926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.3.247.190:43507)
0806 06:51:18.646076 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:18.646085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:18.646114 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:51:18.646239 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:18.646246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:18.646315 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:18.646451 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:18.647213 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.648573 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.649671 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.650288 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.650381 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:18.653215 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:18.653219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:18.653219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:18.653231 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:18.655258 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.655631 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.655634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.655634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.655753 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.655952 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.655953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.655953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.656267 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:18.656417 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:18.656512 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:51:18.656653 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2099,"thread_start_us":57,"threads_started":1}
I20260806 06:51:18.658833 12284 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:18.659312 12284 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:18.663105 12284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Generated new TSK 0
I20260806 06:51:18.663282 12284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:51:18.696484 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:18.787395 12301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:18.787622 12301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:18.787668 12301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:18.788841 12301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:18.788903 12301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:18.788947 12301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:18.790450 12301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:18.790710 12301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:18.790963 12301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:18.796041 12307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.796047 12306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.796072 12309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:18.796392 12301 server_base.cc:1034] running on GCE node
I20260806 06:51:18.796576 12301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:18.796802 12301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:18.797923 12301 hybrid_clock.cc:648] HybridClock initialized: now 1785999078797911 us; error 27 us; skew 500 ppm
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:18.800384 12301 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:51:18.800818 12301 webserver.cc:460] Webserver started at http://127.3.247.129:38919/ using document root <none> and password file <none>
I20260806 06:51:18.801048 12301 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:18.801111 12301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:18.801205 12301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:18.802234 12301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "dc0b073d1cea43e5a003ed92114dd7b9"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "cc6b0dc56d86816528d2a10348b18ba4"
server_key_iv: "6c0a8457fb9d47766c405ef106ce73a8"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.802516 12301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "dc0b073d1cea43e5a003ed92114dd7b9"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "cc6b0dc56d86816528d2a10348b18ba4"
server_key_iv: "6c0a8457fb9d47766c405ef106ce73a8"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.803714 12301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:51:18.804344 12316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.804502 12301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:18.804598 12301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "dc0b073d1cea43e5a003ed92114dd7b9"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "cc6b0dc56d86816528d2a10348b18ba4"
server_key_iv: "6c0a8457fb9d47766c405ef106ce73a8"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.804674 12301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:18.817857 12301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:18.821105 12301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:18.821300 12301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:18.821563 12301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:18.821929 12301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:18.821980 12301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.822021 12301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:18.822048 12301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.827925 12301 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:35345
I20260806 06:51:18.827965 12429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:35345 every 8 connection(s)
I20260806 06:51:18.828306 12301 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:18.831549 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12301
I20260806 06:51:18.831670 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:18.831854 4062 external_mini_cluster.cc:1428] Setting key e64127ef47acab4f02f88b29629ba18e
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:18.838703 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:18.829066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:43317 (local address 127.3.247.190:43507)
0806 06:51:18.829133 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:18.829135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:18.829342 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:51:18.829877 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:18.829878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:18.829894 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:18.829943 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:18.830914 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.831428 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.832842 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.833044 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.833148 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:18.835581 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:18.835582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:18.835583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:18.835590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:18.837288 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.837671 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.837672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.837673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.837740 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.838013 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.838013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.838014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.838162 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:18.838246 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:18.838496 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 06:51:18.838592 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:51:18.838769 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:18.829052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:43317)
0806 06:51:18.829217 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:18.829252 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:18.829755 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:18.830033 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:18.830037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:18.830611 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:18.830806 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:18.830811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.831529 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.831530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:18.832745 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:18.832747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.833155 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.833156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:18.833304 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:18.833896 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:18.833907 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:18.835490 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:18.837376 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:18.837382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:18.837384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:18.837595 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:18.837845 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:18.837846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:18.837846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:18.837944 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:18.838241 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:18.838246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:18.838360 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 06:51:18.838448 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:51:18.838566 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260806 06:51:18.839284 12430 heartbeater.cc:347] Connected to a master server at 127.3.247.190:43507
I20260806 06:51:18.839388 12430 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:18.839584 12430 heartbeater.cc:511] Master 127.3.247.190:43507 requested a full tablet report, sending...
I20260806 06:51:18.840119 12239 ts_manager.cc:194] Registered new tserver with Master: dc0b073d1cea43e5a003ed92114dd7b9 (127.3.247.129:35345)
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:51:18.840804 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:43317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:51:18.856369 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:18.936476 12435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:18.936676 12435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:18.936698 12435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:18.937897 12435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:18.937935 12435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:18.937963 12435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:18.939352 12435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:18.939520 12435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:18.939759 12435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:18.944731 12440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.944738 12443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:18.944746 12441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:18.945058 12435 server_base.cc:1034] running on GCE node
I20260806 06:51:18.945292 12435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:18.945516 12435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:18.946707 12435 hybrid_clock.cc:648] HybridClock initialized: now 1785999078946683 us; error 33 us; skew 500 ppm
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:18.949045 12435 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:51:18.949517 12435 webserver.cc:460] Webserver started at http://127.3.247.130:34683/ using document root <none> and password file <none>
I20260806 06:51:18.949739 12435 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:18.949831 12435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:18.949987 12435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:18.950994 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "e9a0f49d63a649fdaff7415f1a879e19"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "21297aa1b904184fcc2908b7702179c9"
server_key_iv: "9e857f281d2b4892128deaad507bc21f"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.951351 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "e9a0f49d63a649fdaff7415f1a879e19"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "21297aa1b904184fcc2908b7702179c9"
server_key_iv: "9e857f281d2b4892128deaad507bc21f"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.952672 12435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:18.953454 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.953627 12435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:18.953727 12435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "e9a0f49d63a649fdaff7415f1a879e19"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "21297aa1b904184fcc2908b7702179c9"
server_key_iv: "9e857f281d2b4892128deaad507bc21f"
server_key_version: "encryptionkey@0"
I20260806 06:51:18.953840 12435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:18.967538 12435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:18.969246 12435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:18.969406 12435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:18.969622 12435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:18.969939 12435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:18.970002 12435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.970041 12435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:18.970064 12435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:18.976209 12435 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:41613
I20260806 06:51:18.976254 12563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:41613 every 8 connection(s)
I20260806 06:51:18.976572 12435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:18.980813 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12435
I20260806 06:51:18.980930 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:18.981137 4062 external_mini_cluster.cc:1428] Setting key 0b03508b932e3265e603229d5a0b53e3
Aug 06 06:51:18 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999078, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:18.986609 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:18.977241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:47251 (local address 127.3.247.190:43507)
0806 06:51:18.977321 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:18.977324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:18.977514 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:51:18.978022 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:18.978048 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:18.978059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:18.978120 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:18.979034 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.979569 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.980549 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.980741 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.980799 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:18.983503 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:18.983505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:18.983505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:18.983511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:18.985188 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.985601 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.985603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.985603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.985662 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:18.985962 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:18.985963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:18.985963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:18.986079 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:18.986160 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:18.986431 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 06:51:18.986518 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:51:18.986814 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:18.977186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:47251)
0806 06:51:18.977362 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:18.977398 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:18.977926 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:18.978177 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:18.978180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:18.978730 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:18.978923 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:18.978927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.979653 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.979653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:18.980469 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:18.980471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:18.980829 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:18.980830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:18.980990 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:18.981657 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:18.981666 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:18.983403 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:18.985271 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:18.985277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:18.985278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:18.985498 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:18.985732 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:18.985733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:18.985733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:18.985890 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:18.986149 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:18.986155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:18.986298 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 06:51:18.986399 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:51:18.986554 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260806 06:51:18.987310 12564 heartbeater.cc:347] Connected to a master server at 127.3.247.190:43507
I20260806 06:51:18.987418 12564 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:18.987589 12564 heartbeater.cc:511] Master 127.3.247.190:43507 requested a full tablet report, sending...
I20260806 06:51:18.987963 12239 ts_manager.cc:194] Registered new tserver with Master: e9a0f49d63a649fdaff7415f1a879e19 (127.3.247.130:41613)
I20260806 06:51:18.988349 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:47251
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:51:19.006345 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:19.086534 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:19.086728 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:19.086751 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:19.087854 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:19.087888 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:19.087914 12569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:19.089244 12569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:19.089416 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:19.089579 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:19.094427 12574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.094427 12577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.094476 12569 server_base.cc:1034] running on GCE node
W20260806 06:51:19.094470 12575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.094772 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:19.095000 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:19.096160 12569 hybrid_clock.cc:648] HybridClock initialized: now 1785999079096144 us; error 48 us; skew 500 ppm
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:19.098625 12569 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:51:19.099050 12569 webserver.cc:460] Webserver started at http://127.3.247.131:36657/ using document root <none> and password file <none>
I20260806 06:51:19.099265 12569 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:19.099335 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:19.099485 12569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:19.100514 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "fb9184c4175c4002b1ff4f336a7103a1"
format_stamp: "Formatted at 2026-08-06 06:51:19 on dist-test-slave-0kls"
server_key: "b435d2dfe73ebc5285bc14394c7745a2"
server_key_iv: "b39a2eccfb5695e3718454efbd8a93c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.100867 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "fb9184c4175c4002b1ff4f336a7103a1"
format_stamp: "Formatted at 2026-08-06 06:51:19 on dist-test-slave-0kls"
server_key: "b435d2dfe73ebc5285bc14394c7745a2"
server_key_iv: "b39a2eccfb5695e3718454efbd8a93c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.102304 12569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:51:19.103053 12584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.103231 12569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:19.103323 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "fb9184c4175c4002b1ff4f336a7103a1"
format_stamp: "Formatted at 2026-08-06 06:51:19 on dist-test-slave-0kls"
server_key: "b435d2dfe73ebc5285bc14394c7745a2"
server_key_iv: "b39a2eccfb5695e3718454efbd8a93c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.103397 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:19.122354 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:19.124703 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:19.124878 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:19.125135 12569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:19.125453 12569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:19.125487 12569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.125530 12569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:19.125559 12569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.131212 12569 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:39281
I20260806 06:51:19.131295 12697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:39281 every 8 connection(s)
I20260806 06:51:19.131598 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:19.140631 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.132354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:42763 (local address 127.3.247.190:43507)
0806 06:51:19.132415 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.132417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.132619 (+ 202us) server_negotiation.cc:407] Connection header received
0806 06:51:19.133151 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.133153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.133164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.133250 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.134262 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.134759 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.135756 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.135955 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.136061 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.138123 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:19.138125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:19.138125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:19.138132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:19.139560 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:19.139899 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:19.139900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:19.139900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:19.139951 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:19.140160 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:19.140160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:19.140161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:19.140257 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:19.140306 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:19.140510 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 06:51:19.140569 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:51:19.140864 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.132315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:42763)
0806 06:51:19.132496 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.132539 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.133045 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.133332 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.133336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.133968 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.134163 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.134168 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.134856 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.134857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.135644 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.135646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.136068 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.136068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.136198 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.136639 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.136646 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.138038 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:19.139628 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:19.139633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:19.139634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:19.139836 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:19.140005 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:19.140006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:19.140006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:19.140098 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:19.140310 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:19.140314 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:19.140421 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 06:51:19.140499 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:51:19.140608 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260806 06:51:19.141291 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12569
I20260806 06:51:19.141359 12698 heartbeater.cc:347] Connected to a master server at 127.3.247.190:43507
I20260806 06:51:19.141428 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:19.141475 12698 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:19.141637 4062 external_mini_cluster.cc:1428] Setting key 9e1ff8f5cd149678af963e13665d6f88
I20260806 06:51:19.141692 12698 heartbeater.cc:511] Master 127.3.247.190:43507 requested a full tablet report, sending...
I20260806 06:51:19.142167 12239 ts_manager.cc:194] Registered new tserver with Master: fb9184c4175c4002b1ff4f336a7103a1 (127.3.247.131:39281)
I20260806 06:51:19.142639 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:51:19.142684 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:42763
WARNING: no policy specified for oryx/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
Entry for principal oryx/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab.
WARNING: no policy specified for oryx/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
Entry for principal oryx/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab.
WARNING: no policy specified for oryx/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
Entry for principal oryx/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab.
I20260806 06:51:19.208310 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12301
I20260806 06:51:19.212800 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12435
I20260806 06:51:19.216993 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12569
I20260806 06:51:19.220733 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12207
I20260806 06:51:19.225773 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43507
--webserver_interface=127.3.247.190
--webserver_port=37713
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:43507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:19.305819 12707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:19.306032 12707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:19.306056 12707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:19.307274 12707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:19.307322 12707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:19.307335 12707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:19.307348 12707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:19.307360 12707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:19.308831 12707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:43507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:19.309052 12707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:19.309265 12707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:19.314373 12712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.314368 12715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.314368 12713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.314538 12707 server_base.cc:1034] running on GCE node
I20260806 06:51:19.314765 12707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:19.315054 12707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:19.316229 12707 hybrid_clock.cc:648] HybridClock initialized: now 1785999079316217 us; error 27 us; skew 500 ppm
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:19.318717 12707 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:51:19.319196 12707 webserver.cc:460] Webserver started at http://127.3.247.190:37713/ using document root <none> and password file <none>
I20260806 06:51:19.319425 12707 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:19.319509 12707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:19.321341 12707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:51:19.322224 12722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.322422 12707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:19.322556 12707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "9b92d0c380094ad295d8a2a0e0af2a69"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "bbd80af0f992596c4017186a269110e0"
server_key_iv: "6fca308d47381caee2bd780a1e62b7f4"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.323027 12707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:19.349447 12707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:19.351888 12707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:19.352057 12707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:19.355965 12707 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:43507
I20260806 06:51:19.356019 12774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:43507 every 8 connection(s)
I20260806 06:51:19.356412 12707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:19.357928 12775 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:51:19.358537 12775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Bootstrap starting.
I20260806 06:51:19.360726 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12707
I20260806 06:51:19.361065 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:35345
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:43507
--webserver_port=38919
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:51:19.367007 12775 log.cc:824] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:19.368631 12775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:51:19.368928 12775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Bootstrap complete.
I20260806 06:51:19.371234 12775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:19.371449 12775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b92d0c380094ad295d8a2a0e0af2a69, State: Initialized, Role: FOLLOWER
I20260806 06:51:19.371615 12775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:19.371688 12775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:19.371737 12775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:19.371783 12775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:51:19.372812 12775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:19.372921 12775 leader_election.cc:302] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b92d0c380094ad295d8a2a0e0af2a69; no voters:
I20260806 06:51:19.373112 12775 leader_election.cc:288] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:51:19.373148 12779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:51:19.373273 12779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [term 2 LEADER]: Becoming Leader. State: Replica: 9b92d0c380094ad295d8a2a0e0af2a69, State: Running, Role: LEADER
I20260806 06:51:19.373358 12779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } }
I20260806 06:51:19.373386 12775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:19.375007 12780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b92d0c380094ad295d8a2a0e0af2a69. Latest consensus state: current_term: 2 leader_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } } }
I20260806 06:51:19.375093 12780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:19.375419 12781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b92d0c380094ad295d8a2a0e0af2a69" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 43507 } } }
I20260806 06:51:19.375521 12793 catalog_manager.cc:1259] Loaded cluster ID: 762e859d24174ed0b7629c6bb7a399ea
I20260806 06:51:19.375643 12781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:19.375653 12793 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: loading cluster ID for follower catalog manager: success
I20260806 06:51:19.376853 12793 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: acquiring CA information for follower catalog manager: success
I20260806 06:51:19.377179 12793 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:51:19.377291 12795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:19.377429 12795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:19.377506 12795 catalog_manager.cc:1259] Loaded cluster ID: 762e859d24174ed0b7629c6bb7a399ea
I20260806 06:51:19.377535 12795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:19.377697 12795 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:19.377830 12795 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b92d0c380094ad295d8a2a0e0af2a69: Loaded TSK: 0
I20260806 06:51:19.377972 12795 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:51:19.441910 12777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:19.442128 12777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:19.442164 12777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:19.443322 12777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:19.443389 12777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:19.443434 12777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:19.444865 12777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.129.keytab
--principal=oryx/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:35345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=38919
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:19.445122 12777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:19.445346 12777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:19.451395 12801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.451407 12802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.451634 12804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.451702 12777 server_base.cc:1034] running on GCE node
I20260806 06:51:19.451885 12777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:19.452126 12777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:19.453295 12777 hybrid_clock.cc:648] HybridClock initialized: now 1785999079453282 us; error 30 us; skew 500 ppm
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:19.455776 12777 init.cc:377] Logged in from keytab as oryx/127.3.247.129@KRBTEST.COM (short username oryx)
I20260806 06:51:19.456239 12777 webserver.cc:460] Webserver started at http://127.3.247.129:38919/ using document root <none> and password file <none>
I20260806 06:51:19.456480 12777 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:19.456561 12777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:19.457942 12777 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:19.458643 12811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.458822 12777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:51:19.458913 12777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "dc0b073d1cea43e5a003ed92114dd7b9"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "cc6b0dc56d86816528d2a10348b18ba4"
server_key_iv: "6c0a8457fb9d47766c405ef106ce73a8"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.459267 12777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:19.480073 12777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:19.482446 12777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:19.482613 12777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:19.482858 12777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:19.483166 12777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:19.483225 12777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.483274 12777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:19.483302 12777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.489282 12777 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:35345
I20260806 06:51:19.489354 12924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:35345 every 8 connection(s)
I20260806 06:51:19.489704 12777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:19.496142 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12777
I20260806 06:51:19.496479 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:41613
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:43507
--webserver_port=34683
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.499502 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.490495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:38195)
0806 06:51:19.490668 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.490717 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.491176 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.491640 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.491643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.492190 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.492371 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.492375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.494074 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.494075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.494839 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.494841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.495727 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.495728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.495819 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.496404 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.496413 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.499277 (+ 2864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
W20260806 06:51:19.499744 12925 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:43507 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.247.190:43507: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
W20260806 06:51:19.500144 12928 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.490569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:38195 (local address 127.3.247.190:43507)
0806 06:51:19.490769 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.490779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.490822 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:51:19.491385 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.491392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.491449 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.491601 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.492496 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.493973 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.494944 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.495637 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.495751 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.499422 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:38195: BlockingRecv error: recv got EOF from 127.3.247.129:38195 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.505527 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.499961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:58189)
0806 06:51:19.500142 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.500154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.500211 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.500487 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.500488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.500631 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.500682 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.500684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.501665 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.501666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.502382 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.502384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.502901 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.502903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.502966 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.503453 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.503459 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.505478 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162}
W20260806 06:51:19.505676 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.500027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:58189 (local address 127.3.247.190:43507)
0806 06:51:19.500318 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.500321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.500328 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:19.500348 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.500349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.500360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.500456 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.500768 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.501570 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.502471 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.502814 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.502892 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.505593 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:58189: BlockingRecv error: recv got EOF from 127.3.247.129:58189 (error 108)
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":77,"threads_started":1}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:19.510443 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.505953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40013 (local address 127.3.247.190:43507)
0806 06:51:19.505997 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.505999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.506034 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:51:19.506056 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.506057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.506069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.506124 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.506344 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.507106 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.507727 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.508026 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.508093 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.510404 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:40013: BlockingRecv error: recv got EOF from 127.3.247.129:40013 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:51:19.510352 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.505784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:40013)
0806 06:51:19.505946 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.505957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.505997 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.506135 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.506136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.506246 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.506273 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.506274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.507194 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.507194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.507654 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.507655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.508094 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.508095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.508129 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.508539 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.508543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.510284 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
W20260806 06:51:19.510673 12925 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:51:19.581664 12929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:19.581894 12929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:19.581919 12929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:19.583035 12929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:19.583072 12929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:19.583098 12929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:19.584448 12929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.130.keytab
--principal=oryx/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:41613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=34683
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:19.584641 12929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:19.584820 12929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:19.589751 12939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.589896 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.589952 12936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.590183 12929 server_base.cc:1034] running on GCE node
I20260806 06:51:19.590391 12929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:19.590641 12929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:19.591794 12929 hybrid_clock.cc:648] HybridClock initialized: now 1785999079591777 us; error 30 us; skew 500 ppm
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:19.594568 12929 init.cc:377] Logged in from keytab as oryx/127.3.247.130@KRBTEST.COM (short username oryx)
I20260806 06:51:19.594992 12929 webserver.cc:460] Webserver started at http://127.3.247.130:34683/ using document root <none> and password file <none>
I20260806 06:51:19.595194 12929 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:19.595247 12929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:19.596514 12929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:19.597054 12946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.597275 12929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:19.597358 12929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "e9a0f49d63a649fdaff7415f1a879e19"
format_stamp: "Formatted at 2026-08-06 06:51:18 on dist-test-slave-0kls"
server_key: "21297aa1b904184fcc2908b7702179c9"
server_key_iv: "9e857f281d2b4892128deaad507bc21f"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.597669 12929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:19.622433 12929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:19.624764 12929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:19.624876 12929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:19.625101 12929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:19.625398 12929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:19.625429 12929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.625452 12929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:19.625465 12929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.631173 12929 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:41613
I20260806 06:51:19.631258 13059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:41613 every 8 connection(s)
I20260806 06:51:19.631590 12929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:19.632485 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 12929
I20260806 06:51:19.632790 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:39281
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:43507
--webserver_port=36657
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:19.640182 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.632354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:58007 (local address 127.3.247.190:43507)
0806 06:51:19.632427 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.632430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.632667 (+ 237us) server_negotiation.cc:407] Connection header received
0806 06:51:19.633274 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.633276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.633292 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.633370 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.634587 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.635079 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.636654 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.636846 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.636900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.640124 (+ 3224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:58007: BlockingRecv error: recv got EOF from 127.3.247.130:58007 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:51:19.640259 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.632347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:58007)
0806 06:51:19.632525 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.632567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.633155 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.633425 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.633429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.634178 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.634473 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.634481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.635175 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.635176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.636566 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.636569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.636931 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.636932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.637105 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.637608 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.637619 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.640048 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
W20260806 06:51:19.640473 13060 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:43507 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.247.190:43507: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.644836 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.640696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:48465)
0806 06:51:19.640745 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.640755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.640816 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.640935 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.640936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.641065 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.641105 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.641106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.641735 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.641737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.642279 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.642281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.642652 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.642653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.642718 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.643144 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.643149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.644753 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:51:19.644889 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.640764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:48465 (local address 127.3.247.190:43507)
0806 06:51:19.640822 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.640825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.640836 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:19.640868 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.640869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.640880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.640932 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.641191 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.641650 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.642363 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.642568 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.642652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.644825 (+ 2173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:48465: BlockingRecv error: recv got EOF from 127.3.247.130:48465 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.649412 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.645146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:38695)
0806 06:51:19.645197 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.645207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.645266 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.645388 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.645389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.645507 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.645530 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.645531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.646210 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.646211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.646826 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.646829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.647296 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.647296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.647333 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.647753 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.647758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.649342 (+ 1584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:51:19.649502 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.645214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:38695 (local address 127.3.247.190:43507)
0806 06:51:19.645269 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.645271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.645279 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:19.645322 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.645323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.645331 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.645385 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.645603 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.646130 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.646925 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.647225 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.647292 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.649424 (+ 2132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:38695: BlockingRecv error: recv got EOF from 127.3.247.130:38695 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 06:51:19.649629 13060 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:51:19.716509 13063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:19.716745 13063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:19.716781 13063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:19.717995 13063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:19.718061 13063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:19.718109 13063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:19.719564 13063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/krb5kdc/oryx_127.3.247.131.keytab
--principal=oryx/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:39281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=36657
--tserver_master_addrs=127.3.247.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:19.719835 13063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:19.720111 13063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:19.724938 13069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.725064 13071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:19.724951 13068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:19.725167 13063 server_base.cc:1034] running on GCE node
I20260806 06:51:19.725389 13063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:19.725591 13063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:19.726737 13063 hybrid_clock.cc:648] HybridClock initialized: now 1785999079726704 us; error 58 us; skew 500 ppm
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999079, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:19.729368 13063 init.cc:377] Logged in from keytab as oryx/127.3.247.131@KRBTEST.COM (short username oryx)
I20260806 06:51:19.729902 13063 webserver.cc:460] Webserver started at http://127.3.247.131:36657/ using document root <none> and password file <none>
I20260806 06:51:19.730127 13063 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:19.730206 13063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:19.731541 13063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:19.732085 13078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.732272 13063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:19.732366 13063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "fb9184c4175c4002b1ff4f336a7103a1"
format_stamp: "Formatted at 2026-08-06 06:51:19 on dist-test-slave-0kls"
server_key: "b435d2dfe73ebc5285bc14394c7745a2"
server_key_iv: "b39a2eccfb5695e3718454efbd8a93c5"
server_key_version: "encryptionkey@0"
I20260806 06:51:19.732705 13063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:19.750324 13063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:19.752225 13063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:19.752393 13063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:19.752624 13063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:19.752951 13063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:19.753006 13063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.753049 13063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:19.753108 13063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:19.758912 13063 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:39281
I20260806 06:51:19.758957 13191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:39281 every 8 connection(s)
I20260806 06:51:19.759292 13063 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:19.766389 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.760014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:37409 (local address 127.3.247.190:43507)
0806 06:51:19.760060 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.760062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.760268 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:51:19.760726 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.760727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.760737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.760785 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.761760 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.762322 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.763263 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.763446 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.763499 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.766330 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:37409: BlockingRecv error: recv got EOF from 127.3.247.131:37409 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:19.766459 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.760010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:37409)
0806 06:51:19.760147 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.760181 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.760647 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.760843 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.760847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.761467 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.761656 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.761661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.762397 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.762398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.763181 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.763182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.763518 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.763519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.763617 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.764150 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.764158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.766263 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
W20260806 06:51:19.766697 13192 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:43507 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.247.190:43507: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
I20260806 06:51:19.768237 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13063
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.771716 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.766945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:44427)
0806 06:51:19.766992 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.767001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.767037 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.767250 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.767251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.767372 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.767411 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.767412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.768013 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.768014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.768604 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.768608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.769186 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.769189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.769247 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.769692 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.769697 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.771634 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:51:19.771770 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.767100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:44427 (local address 127.3.247.190:43507)
0806 06:51:19.767152 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.767153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.767160 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:19.767177 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.767177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.767185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.767247 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.767485 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.767935 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.768705 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.769005 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.769073 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.771708 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:44427: BlockingRecv error: recv got EOF from 127.3.247.131:44427 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.776149 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.768615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.3.247.190:43507)
0806 06:51:19.768675 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.768677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.768823 (+ 146us) server_negotiation.cc:407] Connection header received
0806 06:51:19.768881 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.768882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.768891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.768933 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.769631 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.770358 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.771356 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.771540 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.771612 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.772857 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:19.772861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:19.772862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:19.772873 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:19.774700 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:19.775020 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:19.775023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:19.775023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:19.775135 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:19.775363 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:19.775363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:19.775364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:19.775648 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:19.775794 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:19.775916 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:51:19.776028 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 06:51:19 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:19.776937 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:19.772083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:57457)
0806 06:51:19.772115 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:19.772125 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:19.772187 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:19.772332 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:19.772332 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:19.772460 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:19.772483 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.772484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.773310 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.773311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.773990 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:19.773992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.774366 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.774367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:19.774423 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.775035 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:19.775039 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:19.776868 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 06:51:19.776994 12930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:19.772136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:57457 (local address 127.3.247.190:43507)
0806 06:51:19.772175 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:19.772178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:19.772187 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:19.772247 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:19.772248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:19.772256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:19.772329 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:19.772556 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.773209 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.774071 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:19.774266 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:19.774350 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:19.776941 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:57457: BlockingRecv error: recv got EOF from 127.3.247.131:57457 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:51:19.777163 13192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:20.515739 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:20.511142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:33181)
0806 06:51:20.511291 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:20.511307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:20.511380 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:20.511515 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:20.511516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:20.511675 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:20.511724 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.511726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.512343 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.512343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.512885 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.512886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.513196 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.513196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.513233 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.513662 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:20.513666 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:20.515654 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260806 06:51:20.515884 13197 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:20.511204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:33181 (local address 127.3.247.190:43507)
0806 06:51:20.511362 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:20.511364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:20.511374 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:20.511426 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:20.511426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:20.511436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:20.511495 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:20.511775 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.512278 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.512954 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.513134 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.513182 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.515764 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:33181: BlockingRecv error: recv got EOF from 127.3.247.129:33181 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:20.654507 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:20.650032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:48053)
0806 06:51:20.650164 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:20.650176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:20.650237 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:20.650350 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:20.650351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:20.650500 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:20.650547 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.650549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.651160 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.651161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.651703 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.651705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.652025 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.652025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.652063 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.652477 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:20.652481 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:20.654396 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260806 06:51:20.654544 13197 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:20.650111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:48053 (local address 127.3.247.190:43507)
0806 06:51:20.650184 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:20.650185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:20.650226 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:51:20.650292 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:20.650292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:20.650302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:20.650339 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:20.650605 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.651092 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.651774 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.651962 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.652012 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.654479 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:48053: BlockingRecv error: recv got EOF from 127.3.247.130:48053 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:20 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:20.782228 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:20.777525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:57811)
0806 06:51:20.777652 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:20.777663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:20.777723 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:20.777899 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:20.777901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:20.778129 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:20.778184 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.778186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.778830 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.778831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.779457 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:20.779459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.779796 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.779797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:20.779844 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.780326 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:20.780331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:20.782154 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
W20260806 06:51:20.782351 13197 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:20.777602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:57811 (local address 127.3.247.190:43507)
0806 06:51:20.777666 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:20.777668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:20.777714 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:51:20.777766 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:20.777766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:20.777776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:20.777843 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:20.778267 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.778751 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.779541 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:20.779716 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:20.779764 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:20.782270 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:57811: BlockingRecv error: recv got EOF from 127.3.247.131:57811 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:21.520586 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:21.516239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:38421)
0806 06:51:21.516380 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:21.516392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:21.516453 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:21.516630 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:21.516631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:21.516785 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:21.516834 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.516835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.517460 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.517460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.517988 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.517989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.518300 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.518301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.518340 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.518851 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:21.518857 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:21.520527 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
W20260806 06:51:21.520709 13201 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:21.516288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:38421 (local address 127.3.247.190:43507)
0806 06:51:21.516417 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:21.516423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:21.516434 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:21.516509 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:21.516509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:21.516520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:21.516580 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:21.516903 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.517391 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.518055 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.518238 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.518292 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.520619 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:38421: BlockingRecv error: recv got EOF from 127.3.247.129:38421 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:21.659333 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:21.654990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:34711)
0806 06:51:21.655103 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:21.655113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:21.655159 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:21.655272 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:21.655273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:21.655425 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:21.655472 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.655473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.656090 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.656091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.656627 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.656629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.657038 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.657040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.657121 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.657543 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:21.657548 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:21.659236 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
W20260806 06:51:21.659384 13201 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:21.655037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:34711 (local address 127.3.247.190:43507)
0806 06:51:21.655141 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:21.655143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:21.655153 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:21.655216 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:21.655217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:21.655227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:21.655269 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:21.655530 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.656022 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.656703 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.656889 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.656992 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.659319 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:34711: BlockingRecv error: recv got EOF from 127.3.247.130:34711 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:21 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:21.787362 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:21.782808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:35259)
0806 06:51:21.782939 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:21.782951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:21.783008 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:21.783150 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:21.783151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:21.783310 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:21.783356 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.783358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.783980 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.783981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.784497 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:21.784498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.784837 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.784838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:21.784887 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.785411 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:21.785422 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:21.787301 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
W20260806 06:51:21.787467 13201 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:21.782935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:35259 (local address 127.3.247.190:43507)
0806 06:51:21.783014 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:21.783016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:21.783028 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:51:21.783062 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:21.783062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:21.783073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:21.783128 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:21.783421 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.783915 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.784568 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:21.784750 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:21.784801 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:21.787393 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:35259: BlockingRecv error: recv got EOF from 127.3.247.131:35259 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:51:22.526065 13205 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:22.521183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:47469 (local address 127.3.247.190:43507)
0806 06:51:22.521314 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:22.521316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:22.521333 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:51:22.521369 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:22.521369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:22.521379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:22.521428 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:22.521741 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.522263 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.522978 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.523161 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.523218 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.526010 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:47469: BlockingRecv error: recv got EOF from 127.3.247.129:47469 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260806 06:51:22.526046 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:22.521073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:47469)
0806 06:51:22.521254 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:22.521265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:22.521323 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:22.521450 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:22.521451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:22.521607 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:22.521657 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.521658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.522345 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.522346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.522898 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.522900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.523285 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.523286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.523345 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.523855 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:22.523859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:22.525922 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:22.664426 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:22.659766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:45159)
0806 06:51:22.659925 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:22.659937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:22.660005 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:22.660144 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:22.660145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:22.660337 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:22.660380 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.660382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.661039 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.661040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.661586 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.661588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.662004 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.662005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.662068 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.662561 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:22.662570 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:22.664324 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
W20260806 06:51:22.664481 13205 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:22.659848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:45159 (local address 127.3.247.190:43507)
0806 06:51:22.659922 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:22.659924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:22.659984 (+ 60us) server_negotiation.cc:407] Connection header received
0806 06:51:22.660068 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:22.660069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:22.660081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:22.660135 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:22.660461 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.660960 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.661671 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.661925 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.661995 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.664415 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:45159: BlockingRecv error: recv got EOF from 127.3.247.130:45159 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:22 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:22.792300 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:22.787857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:52659)
0806 06:51:22.788047 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:22.788062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:22.788127 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:22.788272 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:22.788273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:22.788433 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:22.788479 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.788481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.789121 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.789123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.789676 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:22.789678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.790081 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.790081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:22.790123 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.790536 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:22.790540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:22.792205 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
W20260806 06:51:22.792356 13205 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:22.787941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:52659 (local address 127.3.247.190:43507)
0806 06:51:22.788008 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:22.788010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:22.788116 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:51:22.788189 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:22.788189 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:22.788200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:22.788251 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:22.788541 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.789017 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.789740 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:22.789997 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:22.790049 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:22.792293 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:52659: BlockingRecv error: recv got EOF from 127.3.247.131:52659 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:23.531353 13208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:23.526644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:40635)
0806 06:51:23.526784 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:23.526794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:23.526843 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:23.526982 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:23.526982 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:23.527128 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:23.527178 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.527179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.527957 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.527959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.528538 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.528541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.528878 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.528878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.528918 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.529379 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:23.529384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:23.531252 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
W20260806 06:51:23.531411 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:23.526668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40635 (local address 127.3.247.190:43507)
0806 06:51:23.526817 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:23.526819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:23.526833 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:51:23.526890 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:23.526890 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:23.526900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:23.526968 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:23.527245 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.527861 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.528615 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.528811 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.528863 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.531339 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:40635: BlockingRecv error: recv got EOF from 127.3.247.129:40635 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:23.672415 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:23.664915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:46371)
0806 06:51:23.665046 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:23.665057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:23.665113 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:23.665252 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:23.665252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:23.665409 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:23.665458 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.665459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.666113 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.666114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.666638 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.666639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.666948 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.666949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.666989 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.670717 (+ 3728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:23.670722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:23.672356 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
W20260806 06:51:23.672590 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:23.664989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:46371 (local address 127.3.247.190:43507)
0806 06:51:23.665063 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:23.665065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:23.665098 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:51:23.665154 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:23.665155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:23.665166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:23.665233 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:23.665529 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.666050 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.666708 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.666886 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.666935 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.672486 (+ 5551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:46371: BlockingRecv error: recv got EOF from 127.3.247.130:46371 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:23 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:23.797096 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:23.792811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:38311)
0806 06:51:23.792929 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:23.792944 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:23.793001 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:23.793109 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:23.793110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:23.793256 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:23.793303 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.793305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.793941 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.793942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.794448 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:23.794449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.794759 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.794759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:23.794797 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.795264 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:23.795268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:23.797001 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260806 06:51:23.797147 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:23.792876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:38311 (local address 127.3.247.190:43507)
0806 06:51:23.792943 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:23.792945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:23.793001 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:51:23.793042 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:23.793043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:23.793053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:23.793103 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:23.793358 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.793877 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.794519 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:23.794695 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:23.794755 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:23.797080 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:38311: BlockingRecv error: recv got EOF from 127.3.247.131:38311 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:24.536665 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:24.531957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:53127)
0806 06:51:24.532110 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:24.532129 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:24.532197 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:24.532358 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:24.532359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:24.532517 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:24.532565 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.532566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.533231 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.533232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.533771 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.533772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.534197 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.534198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.534263 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.534697 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:24.534702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:24.536605 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
W20260806 06:51:24.536832 13213 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:24.532017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:53127 (local address 127.3.247.190:43507)
0806 06:51:24.532144 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:24.532146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:24.532177 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:51:24.532260 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:24.532261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:24.532271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:24.532340 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:24.532626 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.533130 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.533927 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.534122 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.534171 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.536725 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:53127: BlockingRecv error: recv got EOF from 127.3.247.129:53127 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:24.677515 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:24.672953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:40509)
0806 06:51:24.673117 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:24.673127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:24.673181 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:24.673310 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:24.673311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:24.673555 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:24.673623 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.673625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.674335 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.674336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.674942 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.674945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.675260 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.675260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.675300 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.675744 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:24.675748 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:24.677455 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
W20260806 06:51:24.677667 13213 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:24.673062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:40509 (local address 127.3.247.190:43507)
0806 06:51:24.673130 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:24.673131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:24.673180 (+ 49us) server_negotiation.cc:407] Connection header received
0806 06:51:24.673229 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:24.673230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:24.673241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:24.673293 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:24.673712 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.674269 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.675014 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.675196 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.675251 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.677579 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:40509: BlockingRecv error: recv got EOF from 127.3.247.130:40509 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:24 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:24.802830 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:24.797629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:55505)
0806 06:51:24.797833 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:24.797846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:24.797940 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:24.798100 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:24.798101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:24.798284 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:24.798329 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.798331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.799053 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.799054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.799617 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:24.799619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.800094 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.800095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:24.800159 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.800580 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:24.800584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:24.802773 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
W20260806 06:51:24.802968 13213 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:24.797735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:55505 (local address 127.3.247.190:43507)
0806 06:51:24.797832 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:24.797835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:24.797919 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:51:24.798006 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:24.798007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:24.798026 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:24.798093 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:24.798416 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.798955 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.799775 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:24.800008 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:24.800087 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:24.802886 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:55505: BlockingRecv error: recv got EOF from 127.3.247.131:55505 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:25.541996 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:25.537330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:42517)
0806 06:51:25.537488 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:25.537498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:25.537554 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:25.537740 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:25.537741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:25.537977 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:25.538026 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.538029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.538659 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.538660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.539179 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.539180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.539502 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.539503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.539543 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.539982 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:25.539987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:25.541934 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
W20260806 06:51:25.542101 13217 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:25.537394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:42517 (local address 127.3.247.190:43507)
0806 06:51:25.537532 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:25.537535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:25.537547 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:51:25.537621 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:25.537622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:25.537632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:25.537701 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:25.538094 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.538590 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.539256 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.539438 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.539486 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.542023 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:42517: BlockingRecv error: recv got EOF from 127.3.247.129:42517 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:25.682654 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:25.678034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:46177)
0806 06:51:25.678198 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:25.678210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:25.678277 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:25.678413 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:25.678414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:25.678574 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:25.678625 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.678627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.679247 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.679248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.679769 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.679771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.680091 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.680091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.680131 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.680588 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:25.680593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:25.682594 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
W20260806 06:51:25.682794 13217 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:25.678124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:46177 (local address 127.3.247.190:43507)
0806 06:51:25.678213 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:25.678215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:25.678276 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:51:25.678326 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:25.678327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:25.678340 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:25.678389 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:25.678685 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.679176 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.679841 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.680024 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.680075 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.682704 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:46177: BlockingRecv error: recv got EOF from 127.3.247.130:46177 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:25 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:25.808125 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:25.803303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:49209)
0806 06:51:25.803500 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:25.803514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:25.803591 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:25.803736 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:25.803737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:25.803913 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:25.803958 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.803959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.804657 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.804658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.805207 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:25.805209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.805590 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.805591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:25.805654 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.806193 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:25.806198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:25.808018 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
W20260806 06:51:25.808163 13217 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:25.803421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:49209 (local address 127.3.247.190:43507)
0806 06:51:25.803504 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:25.803506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:25.803586 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:51:25.803654 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:25.803655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:25.803669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:25.803731 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:25.804069 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.804572 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.805293 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:25.805495 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:25.805588 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:25.808101 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:49209: BlockingRecv error: recv got EOF from 127.3.247.131:49209 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:26.547437 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:26.542641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:44373)
0806 06:51:26.542800 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:26.542812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:26.542896 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:26.543139 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:26.543140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:26.543389 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:26.543471 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.543474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.544161 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.544162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.544695 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.544696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.545019 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.545019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.545059 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.545497 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:26.545501 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:26.547366 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
W20260806 06:51:26.547580 13221 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:26.542734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:44373 (local address 127.3.247.190:43507)
0806 06:51:26.542863 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:26.542865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:26.542888 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:51:26.542986 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:26.542987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:26.543001 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:26.543070 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:26.543556 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.544089 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.544774 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.544959 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.545004 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.547464 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:44373: BlockingRecv error: recv got EOF from 127.3.247.129:44373 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":95,"threads_started":1}
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:26.688130 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:26.683339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:40427)
0806 06:51:26.683608 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:26.683618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:26.683702 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:26.683919 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:26.683920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:26.684143 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:26.684191 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.684194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.684850 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.684851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.685382 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.685383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.685720 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.685721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.685763 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.686274 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:26.686279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:26.688062 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":153,"threads_started":1}
W20260806 06:51:26.688267 13221 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:26.683540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:40427 (local address 127.3.247.190:43507)
0806 06:51:26.683600 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:26.683602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:26.683716 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:51:26.683772 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:26.683773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:26.683787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:26.683879 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:26.684283 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.684784 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.685459 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.685653 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.685707 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.688185 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:40427: BlockingRecv error: recv got EOF from 127.3.247.130:40427 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:26 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:26.813139 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:26.808675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:42019)
0806 06:51:26.808854 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:26.808866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:26.808928 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:26.809043 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:26.809044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:26.809212 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:26.809260 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.809262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.809893 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.809894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.810427 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:26.810429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.810745 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.810746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:26.810790 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.811311 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:26.811323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:26.813080 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
W20260806 06:51:26.813304 13221 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:26.808744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:42019 (local address 127.3.247.190:43507)
0806 06:51:26.808813 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:26.808815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:26.808940 (+ 125us) server_negotiation.cc:407] Connection header received
0806 06:51:26.808967 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:26.808968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:26.808978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:26.809028 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:26.809323 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.809823 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.810499 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:26.810674 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:26.810722 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:26.813165 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:42019: BlockingRecv error: recv got EOF from 127.3.247.131:42019 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:27.552420 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:27.547926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:39721)
0806 06:51:27.548065 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:27.548076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:27.548126 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:27.548312 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:27.548313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:27.548468 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:27.548518 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.548519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.549176 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.549176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.549679 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.549680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.550033 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.550034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.550074 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.550562 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:27.550567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:27.552306 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
W20260806 06:51:27.552495 13225 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:27.548049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:39721 (local address 127.3.247.190:43507)
0806 06:51:27.548192 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:27.548194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:27.548201 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:27.548222 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:27.548223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:27.548232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:27.548294 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:27.548583 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.549088 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.549760 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.549971 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.550024 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.552412 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:39721: BlockingRecv error: recv got EOF from 127.3.247.129:39721 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:27.693517 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:27.688727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:36981)
0806 06:51:27.688914 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:27.688925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:27.688986 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:27.689232 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:27.689234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:27.689394 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:27.689439 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.689441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.690226 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.690227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.690737 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.690738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.691036 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.691037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.691098 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.691620 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:27.691624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:27.693426 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":124,"threads_started":1}
W20260806 06:51:27.693550 13225 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:27.688778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:36981 (local address 127.3.247.190:43507)
0806 06:51:27.688874 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:27.688876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:27.688988 (+ 112us) server_negotiation.cc:407] Connection header received
0806 06:51:27.689089 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:27.689090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:27.689101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:27.689165 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:27.689506 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.690159 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.690807 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.690978 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.691024 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.693502 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:36981: BlockingRecv error: recv got EOF from 127.3.247.130:36981 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:27 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:27.818454 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:27.813598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:54833)
0806 06:51:27.813803 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:27.813817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:27.813908 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:27.814050 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:27.814051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:27.814251 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:27.814295 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.814297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.814975 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.814976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.815531 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:27.815533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.815908 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.815909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:27.815970 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.816478 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:27.816483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:27.818335 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
W20260806 06:51:27.818490 13225 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:27.813679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:54833 (local address 127.3.247.190:43507)
0806 06:51:27.813766 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:27.813769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:27.813889 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:51:27.813981 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:27.813982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:27.813995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:27.814046 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:27.814378 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.814882 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.815623 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:27.815833 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:27.815910 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:27.818421 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:54833: BlockingRecv error: recv got EOF from 127.3.247.131:54833 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:28.557221 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:28.552870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:40485)
0806 06:51:28.552980 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:28.552993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:28.553060 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:28.553170 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:28.553170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:28.553335 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:28.553382 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.553384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.554074 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.554075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.554591 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.554592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.554914 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.554915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.554951 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.555387 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:28.555391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:28.557152 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":53,"threads_started":1}
W20260806 06:51:28.557343 13229 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:28.552941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40485 (local address 127.3.247.190:43507)
0806 06:51:28.553067 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:28.553069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:28.553079 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:28.553101 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:28.553102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:28.553111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:28.553165 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:28.553444 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.554012 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.554660 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.554854 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.554902 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.557265 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:40485: BlockingRecv error: recv got EOF from 127.3.247.129:40485 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:28.702014 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:28.694011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:60497)
0806 06:51:28.694151 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:28.694167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:28.694225 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:28.694356 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:28.694357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:28.694513 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:28.694569 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.694570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.695202 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.695203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.695743 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.695744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.696123 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.696124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.696164 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.700108 (+ 3944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:28.700113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:28.701956 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
W20260806 06:51:28.702134 13229 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:28.694084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:60497 (local address 127.3.247.190:43507)
0806 06:51:28.694152 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:28.694154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:28.694226 (+ 72us) server_negotiation.cc:407] Connection header received
0806 06:51:28.694279 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:28.694279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:28.694290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:28.694330 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:28.694630 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.695131 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.695847 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.696036 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.696087 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.702053 (+ 5966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:60497: BlockingRecv error: recv got EOF from 127.3.247.130:60497 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:28 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:28.823603 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:28.819070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:36615)
0806 06:51:28.819214 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:28.819230 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:28.819288 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:28.819423 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:28.819424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:28.819582 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:28.819630 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.819631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.820261 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.820262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.820797 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:28.820798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.821137 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.821138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:28.821184 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.821615 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:28.821620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:28.823503 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260806 06:51:28.823645 13229 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:28.819141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36615 (local address 127.3.247.190:43507)
0806 06:51:28.819224 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:28.819226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:28.819290 (+ 64us) server_negotiation.cc:407] Connection header received
0806 06:51:28.819348 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:28.819349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:28.819362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:28.819406 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:28.819693 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.820187 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.820884 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:28.821069 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:28.821122 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:28.823582 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:36615: BlockingRecv error: recv got EOF from 127.3.247.131:36615 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:29.562927 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:29.557887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:53357)
0806 06:51:29.558068 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:29.558081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:29.558158 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:29.558347 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:29.558348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:29.558607 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:29.558667 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.558669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.559447 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.559448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.559999 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.560001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.560390 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.560390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.560430 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.560893 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:29.560898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:29.562866 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
W20260806 06:51:29.563053 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:29.557936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:53357 (local address 127.3.247.190:43507)
0806 06:51:29.558111 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:29.558113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:29.558166 (+ 53us) server_negotiation.cc:407] Connection header received
0806 06:51:29.558225 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:29.558226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:29.558242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:29.558324 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:29.558754 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.559351 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.560111 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.560318 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.560379 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.562962 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:53357: BlockingRecv error: recv got EOF from 127.3.247.129:53357 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":153,"threads_started":1}
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:29.706802 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:29.702476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:34733)
0806 06:51:29.702606 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:29.702617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:29.702671 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:29.702813 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:29.702814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:29.702973 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:29.703020 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.703021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.703625 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.703626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.704136 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.704137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.704444 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.704445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.704483 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.704916 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:29.704921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:29.706698 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
W20260806 06:51:29.706863 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:29.702549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:34733 (local address 127.3.247.190:43507)
0806 06:51:29.702600 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:29.702602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:29.702659 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:51:29.702729 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:29.702729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:29.702739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:29.702794 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:29.703083 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.703563 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.704204 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.704382 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.704430 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.706802 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:34733: BlockingRecv error: recv got EOF from 127.3.247.130:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:29 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:29.828770 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:29.824189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:42885)
0806 06:51:29.824338 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:29.824353 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:29.824405 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:29.824552 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:29.824552 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:29.824709 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:29.824758 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.824759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.825484 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.825485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.826070 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:29.826073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.826477 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.826478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:29.826518 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.827011 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:29.827016 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:29.828701 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260806 06:51:29.828920 13233 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:29.824339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:42885 (local address 127.3.247.190:43507)
0806 06:51:29.824417 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:29.824419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:29.824445 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:51:29.824477 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:29.824478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:29.824488 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:29.824532 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:29.824826 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.825394 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.826175 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:29.826365 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:29.826476 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:29.828820 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:42885: BlockingRecv error: recv got EOF from 127.3.247.131:42885 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:30.568431 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:30.563475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:44043)
0806 06:51:30.563622 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:30.563632 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:30.563685 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:30.563844 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:30.563845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:30.564007 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:30.564051 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.564053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.564954 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.564955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.565530 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.565532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.565943 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.565944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.566006 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.566529 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:30.566534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:30.568321 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
W20260806 06:51:30.568480 13237 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:30.563556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:44043 (local address 127.3.247.190:43507)
0806 06:51:30.563711 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:30.563713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:30.563723 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:30.563747 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:30.563747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:30.563757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:30.563827 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:30.564130 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.564844 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.565626 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.565856 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.565931 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.568404 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:44043: BlockingRecv error: recv got EOF from 127.3.247.129:44043 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:30.711990 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:30.707313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:54859)
0806 06:51:30.707484 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:30.707497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:30.707584 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:30.707712 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:30.707712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:30.707901 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:30.707943 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.707945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.708649 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.708650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.709190 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.709191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.709606 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.709607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.709668 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.710202 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:30.710207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:30.711882 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
W20260806 06:51:30.712031 13237 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:30.707393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:54859 (local address 127.3.247.190:43507)
0806 06:51:30.707464 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:30.707466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:30.707568 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:51:30.707643 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:30.707644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:30.707656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:30.707707 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:30.708029 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.708555 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.709309 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.709518 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.709606 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.711965 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:54859: BlockingRecv error: recv got EOF from 127.3.247.130:54859 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:30 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:30.834443 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:30.829282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:36601)
0806 06:51:30.829468 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:30.829482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:30.829598 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:30.829803 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:30.829806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:30.830005 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:30.830084 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.830086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.830800 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.830801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.831377 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:30.831379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.831781 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.831782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:30.831843 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.832329 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:30.832333 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:30.834322 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
W20260806 06:51:30.834477 13237 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:30.829375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:36601 (local address 127.3.247.190:43507)
0806 06:51:30.829472 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:30.829474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:30.829576 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:51:30.829663 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:30.829664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:30.829681 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:30.829758 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:30.830194 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.830709 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.831476 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:30.831686 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:30.831779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:30.834407 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:36601: BlockingRecv error: recv got EOF from 127.3.247.131:36601 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:31.573932 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:31.569077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:39769)
0806 06:51:31.569250 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:31.569261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:31.569316 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:31.569574 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:31.569576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:31.569899 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:31.569965 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.569968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.570641 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.570642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.571174 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.571176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.571494 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.571495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.571533 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.571977 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:31.571982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:31.573873 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
W20260806 06:51:31.574074 13241 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:31.569171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:39769 (local address 127.3.247.190:43507)
0806 06:51:31.569302 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:31.569311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:31.569321 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:31.569418 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:31.569419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:31.569432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:31.569505 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:31.570056 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.570555 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.571247 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.571432 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.571481 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.573959 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:39769: BlockingRecv error: recv got EOF from 127.3.247.129:39769 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:31.717073 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:31.712506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:58193)
0806 06:51:31.712677 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:31.712691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:31.712762 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:31.712931 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:31.712932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:31.713110 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:31.713153 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.713155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.713870 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.713871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.714416 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.714418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.714778 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.714779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.714831 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.715289 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:31.715293 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:31.716965 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
W20260806 06:51:31.717114 13241 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:31.712621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:58193 (local address 127.3.247.190:43507)
0806 06:51:31.712691 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:31.712693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:31.712757 (+ 64us) server_negotiation.cc:407] Connection header received
0806 06:51:31.712829 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:31.712830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:31.712843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:31.712909 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:31.713235 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.713732 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.714503 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.714701 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.714787 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.717049 (+ 2262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:58193: BlockingRecv error: recv got EOF from 127.3.247.130:58193 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:31 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:31.839773 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:31.834979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:47725)
0806 06:51:31.835100 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:31.835128 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:31.835218 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:31.835427 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:31.835428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:31.835614 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:31.835658 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.835660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.836393 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.836394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.836935 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:31.836938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.837298 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.837299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:31.837365 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.837846 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:31.837850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:31.839699 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
W20260806 06:51:31.839905 13241 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:31.835116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:47725 (local address 127.3.247.190:43507)
0806 06:51:31.835191 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:31.835193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:31.835203 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:51:31.835334 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:31.835335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:31.835347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:31.835420 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:31.835748 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.836306 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.837030 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:31.837220 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:31.837299 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:31.839823 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:47725: BlockingRecv error: recv got EOF from 127.3.247.131:47725 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:32.578912 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:32.574382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:37059)
0806 06:51:32.574510 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:32.574521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:32.574603 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:32.574770 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:32.574771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:32.574999 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:32.575049 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.575051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.575678 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.575679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.576223 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.576225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.576544 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.576545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.576585 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.576992 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:32.576996 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:32.578839 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
W20260806 06:51:32.579022 13245 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:32.574465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:37059 (local address 127.3.247.190:43507)
0806 06:51:32.574600 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:32.574602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:32.574609 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:51:32.574669 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:32.574669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:32.574682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:32.574760 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:32.575116 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.575612 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.576300 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.576483 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.576532 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.578941 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:37059: BlockingRecv error: recv got EOF from 127.3.247.129:37059 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:32.722411 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:32.717652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:37851)
0806 06:51:32.717862 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:32.717875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:32.717959 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:32.718107 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:32.718108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:32.718293 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:32.718336 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.718337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.719001 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.719003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.719536 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.719538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.719912 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.719913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.719975 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.720508 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:32.720512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:32.722352 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":147,"threads_started":1}
W20260806 06:51:32.722540 13245 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:32.717778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:37851 (local address 127.3.247.190:43507)
0806 06:51:32.717858 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:32.717861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:32.717941 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:51:32.718018 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:32.718019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:32.718031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:32.718103 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:32.718420 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.718918 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.719630 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.719827 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.719910 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.722459 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:37851: BlockingRecv error: recv got EOF from 127.3.247.130:37851 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:32 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:32.844848 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:32.840336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:44675)
0806 06:51:32.840466 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:32.840477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:32.840559 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:32.840687 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:32.840688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:32.840858 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:32.840904 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.840906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.841549 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.841550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.842199 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:32.842201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.842523 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.842524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:32.842563 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.842988 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:32.842993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:32.844791 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":55,"threads_started":1}
W20260806 06:51:32.844986 13245 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:32.840405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:44675 (local address 127.3.247.190:43507)
0806 06:51:32.840497 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:32.840499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:32.840559 (+ 60us) server_negotiation.cc:407] Connection header received
0806 06:51:32.840607 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:32.840608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:32.840620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:32.840672 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:32.840967 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.841468 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.842276 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:32.842455 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:32.842509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:32.844903 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:44675: BlockingRecv error: recv got EOF from 127.3.247.131:44675 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:33.583886 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:33.579412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:40747)
0806 06:51:33.579565 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:33.579578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:33.579639 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:33.579786 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:33.579787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:33.579950 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:33.579997 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.579998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.580615 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.580616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.581122 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.581123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.581436 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.581437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.581475 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.581943 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:33.581947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:33.583828 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
W20260806 06:51:33.583994 13249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:33.579465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40747 (local address 127.3.247.190:43507)
0806 06:51:33.579613 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:33.579614 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:33.579625 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:33.579698 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:33.579698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:33.579708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:33.579767 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:33.580065 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.580549 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.581191 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.581373 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.581421 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.583915 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:40747: BlockingRecv error: recv got EOF from 127.3.247.129:40747 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:33.731664 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:33.722897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:39335)
0806 06:51:33.723072 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:33.723085 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:33.723181 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:33.723379 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:33.723380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:33.723616 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:33.723660 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.723661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.724350 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.724351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.724921 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.724922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.725290 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.725292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.725360 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.729493 (+ 4133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:33.729498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:33.731554 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
W20260806 06:51:33.731711 13249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:33.722990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:39335 (local address 127.3.247.190:43507)
0806 06:51:33.723065 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:33.723068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:33.723162 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:51:33.723241 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:33.723242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:33.723261 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:33.723308 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:33.723750 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.724264 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.725008 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.725207 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.725281 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.731645 (+ 6364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:39335: BlockingRecv error: recv got EOF from 127.3.247.130:39335 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:33 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.131@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:33.850040 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:33.845353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.131:53985)
0806 06:51:33.845504 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:33.845516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:33.845581 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:33.845709 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:33.845710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:33.845893 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:33.845937 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.845939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.846553 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.846554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.847072 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:33.847073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.847378 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.847379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:33.847417 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.847846 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:33.847850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:33.849969 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
W20260806 06:51:33.850143 13249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:33.845445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:53985 (local address 127.3.247.190:43507)
0806 06:51:33.845499 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:33.845500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:33.845569 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:51:33.845625 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:33.845625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:33.845637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:33.845690 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:33.846010 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.846488 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.847142 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:33.847316 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:33.847363 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:33.850070 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.131:53985: BlockingRecv error: recv got EOF from 127.3.247.131:53985 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:51:34 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:34 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.129@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:34.589162 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:34.584442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.129:33209)
0806 06:51:34.584581 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:34.584593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:34.584655 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:34.584798 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:34.584798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:34.584959 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:34.585007 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:34.585008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.585640 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.585641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:34.586252 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:34.586254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.586610 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.586611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:34.586668 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:34.587314 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:34.587320 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:34.589101 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
W20260806 06:51:34.589269 13253 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:34.584525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:33209 (local address 127.3.247.190:43507)
0806 06:51:34.584665 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:34.584667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:34.584675 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:51:34.584700 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:34.584700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:34.584710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:34.584776 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:34.585074 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.585571 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.586341 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.586534 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.586590 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:34.589193 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.129:33209: BlockingRecv error: recv got EOF from 127.3.247.129:33209 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
Aug 06 06:51:34 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:51:34 dist-test-slave-0kls krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.247.130@KRBTEST.COM for oryx/127.3.247.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:51:34.736840 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:34.732215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:43507 (local address 127.3.247.130:36021)
0806 06:51:34.732352 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:34.732363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:34.732429 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:34.732553 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:34.732553 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:34.732704 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:34.732751 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:34.732753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.733365 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.733366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:34.733946 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:34.733948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.734277 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.734278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:34.734330 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:34.734771 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:34.734775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:34.736746 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.247.190:43507: Server oryx/127.3.247.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
W20260806 06:51:34.736907 13253 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:34.732284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:36021 (local address 127.3.247.190:43507)
0806 06:51:34.732352 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:34.732354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:34.732429 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:51:34.732473 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:34.732474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:34.732483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:34.732537 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:34.732825 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.733301 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.734009 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.734185 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.734235 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:34.736841 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.247.130:36021: BlockingRecv error: recv got EOF from 127.3.247.130:36021 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:51:34.771270 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12777
I20260806 06:51:34.775902 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12929
I20260806 06:51:34.780084 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13063
I20260806 06:51:34.784116 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 12707
2026-08-06T06:51:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17430 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T06:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:34Z Disabled control of system clock
I20260806 06:51:34.811738 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38343
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:51:34.891863 13258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:34.892076 13258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:34.892097 13258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:34.893204 13258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:34.893245 13258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:34.893260 13258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:34.893275 13258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:34.893289 13258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:34.894646 13258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:34.894840 13258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:34.895076 13258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:51:34.900574 13258 server_base.cc:1034] running on GCE node
W20260806 06:51:34.900516 13266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:34.900542 13264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:34.900556 13263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:34.900977 13258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:34.901264 13258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:34.902426 13258 hybrid_clock.cc:648] HybridClock initialized: now 1785999094902400 us; error 53 us; skew 500 ppm
I20260806 06:51:34.903653 13258 webserver.cc:460] Webserver started at http://127.3.247.190:44199/ using document root <none> and password file <none>
I20260806 06:51:34.903893 13258 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:34.903939 13258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:34.904079 13258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:34.905134 13258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "1c136955e5024b5a9de73f98fd6e9ef7"
format_stamp: "Formatted at 2026-08-06 06:51:34 on dist-test-slave-0kls"
server_key: "89d8497a9f14564d9ecbf19ba16ce10b"
server_key_iv: "1dc6837de943840d2ca09337f6337ceb"
server_key_version: "encryptionkey@0"
I20260806 06:51:34.905504 13258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "1c136955e5024b5a9de73f98fd6e9ef7"
format_stamp: "Formatted at 2026-08-06 06:51:34 on dist-test-slave-0kls"
server_key: "89d8497a9f14564d9ecbf19ba16ce10b"
server_key_iv: "1dc6837de943840d2ca09337f6337ceb"
server_key_version: "encryptionkey@0"
I20260806 06:51:34.906855 13258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:34.907625 13272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:34.907819 13258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:34.907908 13258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "1c136955e5024b5a9de73f98fd6e9ef7"
format_stamp: "Formatted at 2026-08-06 06:51:34 on dist-test-slave-0kls"
server_key: "89d8497a9f14564d9ecbf19ba16ce10b"
server_key_iv: "1dc6837de943840d2ca09337f6337ceb"
server_key_version: "encryptionkey@0"
I20260806 06:51:34.907966 13258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:34.933732 13258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:34.934115 13258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:34.934238 13258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:34.938591 13258 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38343
I20260806 06:51:34.938652 13324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38343 every 8 connection(s)
I20260806 06:51:34.939009 13258 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:34.939743 13325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:34.942415 13325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7: Bootstrap starting.
I20260806 06:51:34.942997 13325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:34.943297 13325 log.cc:824] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:34.943994 13325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7: No bootstrap required, opened a new log
I20260806 06:51:34.945529 13325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } }
I20260806 06:51:34.945667 13325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:34.945690 13325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c136955e5024b5a9de73f98fd6e9ef7, State: Initialized, Role: FOLLOWER
I20260806 06:51:34.945876 13325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } }
I20260806 06:51:34.945948 13325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:34.945969 13325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:34.946034 13325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:34.946265 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13258
I20260806 06:51:34.946348 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:34.946590 4062 external_mini_cluster.cc:1428] Setting key a3f26350b53e7c67b4e1dbb18b46cb21
I20260806 06:51:34.946655 13325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } }
I20260806 06:51:34.946764 13325 leader_election.cc:302] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c136955e5024b5a9de73f98fd6e9ef7; no voters:
I20260806 06:51:34.946923 13325 leader_election.cc:288] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:34.946964 13328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:34.947089 13328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [term 1 LEADER]: Becoming Leader. State: Replica: 1c136955e5024b5a9de73f98fd6e9ef7, State: Running, Role: LEADER
I20260806 06:51:34.947180 13325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:34.947243 13328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } }
I20260806 06:51:34.947592 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c136955e5024b5a9de73f98fd6e9ef7. Latest consensus state: current_term: 1 leader_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } } }
I20260806 06:51:34.947602 13329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c136955e5024b5a9de73f98fd6e9ef7" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38343 } } }
I20260806 06:51:34.947844 13329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:34.947757 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:34.948165 13334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:34.948724 13334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:34.950835 13334 catalog_manager.cc:1347] Generated new cluster ID: a3fdb8db0bf345bdb6e3330ce7ed7b04
I20260806 06:51:34.950906 13334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:34.953442 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:34.947356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.3.247.190:38343)
0806 06:51:34.947606 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:34.947615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:34.947651 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:51:34.947765 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:34.947772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:34.947841 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:34.947994 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:34.948368 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.949760 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.950780 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:34.951464 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:34.951583 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:34.952538 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:34.952542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:34.952542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:34.952748 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:34.952788 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:34.952801 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:34.952877 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:34.953149 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 06:51:34.953224 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260806 06:51:34.959740 13334 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:34.960413 13334 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:34.971956 13334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c136955e5024b5a9de73f98fd6e9ef7: Generated new TSK 0
I20260806 06:51:34.972141 13334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:34.979070 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--builtin_ntp_servers=127.3.247.148:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:51:35.062171 13349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.062391 13349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.062427 13349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.063643 13349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.063699 13349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.063741 13349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:35.065060 13349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.065330 13349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.065618 13349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.071319 13357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.071301 13354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.071588 13349 server_base.cc:1034] running on GCE node
W20260806 06:51:35.071322 13355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.071862 13349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.072113 13349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.073256 13349 hybrid_clock.cc:648] HybridClock initialized: now 1785999095073254 us; error 39 us; skew 500 ppm
I20260806 06:51:35.074455 13349 webserver.cc:460] Webserver started at http://127.3.247.129:41047/ using document root <none> and password file <none>
I20260806 06:51:35.074702 13349 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.074747 13349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.074940 13349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.075985 13349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "b01a8570a5394d8580aa2515ea2b3688"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "efbf69339510b4c2695e8d0eff3499b2"
server_key_iv: "7a043336aab5212510c68e2f112d9022"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.076344 13349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "b01a8570a5394d8580aa2515ea2b3688"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "efbf69339510b4c2695e8d0eff3499b2"
server_key_iv: "7a043336aab5212510c68e2f112d9022"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.077651 13349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:35.078416 13363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.078605 13349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.078706 13349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "b01a8570a5394d8580aa2515ea2b3688"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "efbf69339510b4c2695e8d0eff3499b2"
server_key_iv: "7a043336aab5212510c68e2f112d9022"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.078799 13349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.090288 13349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.090569 13349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.090719 13349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.090934 13349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.091251 13349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.091320 13349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.091355 13349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.091382 13349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.097291 13349 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:46749
I20260806 06:51:35.097344 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:46749 every 8 connection(s)
I20260806 06:51:35.097648 13349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:35.102511 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.098351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40701 (local address 127.3.247.190:38343)
0806 06:51:35.098415 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.098417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.098720 (+ 303us) server_negotiation.cc:407] Connection header received
0806 06:51:35.099177 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.099178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.099190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.099242 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.099829 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.100318 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.101270 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.101455 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.101507 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.102289 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.102290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.102290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.102295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.102309 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.102313 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.102365 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.102425 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.102451 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:51:35.102722 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.098430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38343 (local address 127.3.247.129:40701)
0806 06:51:35.098603 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.098634 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.099097 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.099294 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.099298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.099305 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.099756 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.099760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.100395 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.100396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.101182 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.101184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.101529 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.101530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.101632 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.102156 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.102157 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.102183 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.102204 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.102367 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.102369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.102371 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.102415 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.102487 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260806 06:51:35.103247 13477 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38343
I20260806 06:51:35.103332 13477 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.103468 13477 heartbeater.cc:511] Master 127.3.247.190:38343 requested a full tablet report, sending...
I20260806 06:51:35.103965 13289 ts_manager.cc:194] Registered new tserver with Master: b01a8570a5394d8580aa2515ea2b3688 (127.3.247.129:46749)
I20260806 06:51:35.104130 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13349
I20260806 06:51:35.104212 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:35.104409 4062 external_mini_cluster.cc:1428] Setting key c5954319bf3a9ee84374a724d51eb398
I20260806 06:51:35.104898 13289 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.129:40701
I20260806 06:51:35.105381 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--builtin_ntp_servers=127.3.247.148:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:51:35.184592 13480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.184852 13480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.184890 13480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.186206 13480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.186277 13480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.186322 13480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:35.187685 13480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.187963 13480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.188265 13480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.193426 13486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.193427 13485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.193473 13480 server_base.cc:1034] running on GCE node
W20260806 06:51:35.193621 13488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.193871 13480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.194113 13480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.195251 13480 hybrid_clock.cc:648] HybridClock initialized: now 1785999095195233 us; error 33 us; skew 500 ppm
I20260806 06:51:35.196352 13480 webserver.cc:460] Webserver started at http://127.3.247.130:36739/ using document root <none> and password file <none>
I20260806 06:51:35.196547 13480 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.196586 13480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.196673 13480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.197677 13480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "69d714731b4a410ca3c84c628aeebcd9"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "e5c06a7ffd1dd3374328e6e199306eff"
server_key_iv: "e404c83ccc29777d39b4418dd08c9448"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.198052 13480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "69d714731b4a410ca3c84c628aeebcd9"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "e5c06a7ffd1dd3374328e6e199306eff"
server_key_iv: "e404c83ccc29777d39b4418dd08c9448"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.199259 13480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:51:35.200006 13494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.200153 13480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:35.200235 13480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "69d714731b4a410ca3c84c628aeebcd9"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "e5c06a7ffd1dd3374328e6e199306eff"
server_key_iv: "e404c83ccc29777d39b4418dd08c9448"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.200296 13480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.224828 13480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.225113 13480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.225220 13480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.225405 13480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.225775 13480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.225843 13480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.225867 13480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.225880 13480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.231462 13480 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:39385
I20260806 06:51:35.231528 13607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:39385 every 8 connection(s)
I20260806 06:51:35.231868 13480 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:35.236867 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.232623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:47021 (local address 127.3.247.190:38343)
0806 06:51:35.232685 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.232687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.232937 (+ 250us) server_negotiation.cc:407] Connection header received
0806 06:51:35.233414 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.233415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.233425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.233473 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.234187 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.234671 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.235631 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.235819 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.235869 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.236622 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.236623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.236624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.236629 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.236642 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.236646 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.236687 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.236772 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.236805 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:51:35.237046 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.232599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38343 (local address 127.3.247.130:47021)
0806 06:51:35.232794 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.232839 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.233329 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.233536 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.233539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.233547 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.234109 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.234114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.234754 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.234755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.235551 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.235553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.235894 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.235894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.236001 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.236488 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.236488 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.236518 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.236541 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.236700 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.236704 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.236706 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.236760 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.236838 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260806 06:51:35.237545 13608 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38343
I20260806 06:51:35.237658 13608 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.237885 13608 heartbeater.cc:511] Master 127.3.247.190:38343 requested a full tablet report, sending...
I20260806 06:51:35.238267 13289 ts_manager.cc:194] Registered new tserver with Master: 69d714731b4a410ca3c84c628aeebcd9 (127.3.247.130:39385)
I20260806 06:51:35.238606 13289 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.130:47021
I20260806 06:51:35.240049 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13480
I20260806 06:51:35.240124 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:35.240288 4062 external_mini_cluster.cc:1428] Setting key cfea4055d737f91d6902cccbb31a44d5
I20260806 06:51:35.241111 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--builtin_ntp_servers=127.3.247.148:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:51:35.321251 13611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.321504 13611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.321542 13611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.322764 13611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.322841 13611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.322885 13611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:35.324208 13611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.324487 13611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.324802 13611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.329826 13616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.329820 13619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.329831 13617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.330155 13611 server_base.cc:1034] running on GCE node
I20260806 06:51:35.330356 13611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.330577 13611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.331774 13611 hybrid_clock.cc:648] HybridClock initialized: now 1785999095331752 us; error 38 us; skew 500 ppm
I20260806 06:51:35.333050 13611 webserver.cc:460] Webserver started at http://127.3.247.131:35839/ using document root <none> and password file <none>
I20260806 06:51:35.333298 13611 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.333344 13611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.333513 13611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.334632 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "bc091c207cb14a4b918ab2f586aa6aba"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "2d5c4897a4f5aad8b03a5725b4c55a57"
server_key_iv: "613a0a199a51d7e25a0bd5bfdfab26da"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.334983 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "bc091c207cb14a4b918ab2f586aa6aba"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "2d5c4897a4f5aad8b03a5725b4c55a57"
server_key_iv: "613a0a199a51d7e25a0bd5bfdfab26da"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.336256 13611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:51:35.336941 13625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.337116 13611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:35.337204 13611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "bc091c207cb14a4b918ab2f586aa6aba"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "2d5c4897a4f5aad8b03a5725b4c55a57"
server_key_iv: "613a0a199a51d7e25a0bd5bfdfab26da"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.337275 13611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.355844 13611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.356161 13611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.356310 13611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.356541 13611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.356873 13611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.356907 13611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.356930 13611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.356977 13611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.362787 13611 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:35475
I20260806 06:51:35.362835 13738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:35475 every 8 connection(s)
I20260806 06:51:35.363160 13611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:35.365494 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13611
I20260806 06:51:35.365582 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:35.365777 4062 external_mini_cluster.cc:1428] Setting key 077662bd8edf80f29a107d0f9eef707d
I20260806 06:51:35.368492 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.363891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:46687 (local address 127.3.247.190:38343)
0806 06:51:35.363971 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.363973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.364165 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:51:35.364654 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.364656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.364665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.364717 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.365407 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.366156 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.367312 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.367496 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.367550 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.368290 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.368291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.368292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.368297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.368310 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.368314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.368359 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.368413 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.368436 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:51:35.368750 13741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.363845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38343 (local address 127.3.247.131:46687)
0806 06:51:35.364047 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.364083 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.364573 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.364795 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.364799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.364808 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.365326 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.365330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.366263 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.366265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.367231 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:35.367234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:35.367568 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:35.367568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:35.367674 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:35.368155 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.368155 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.368187 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.368209 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.368362 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.368365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.368367 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.368413 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.368489 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260806 06:51:35.369211 13739 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38343
I20260806 06:51:35.369293 13739 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.369429 13739 heartbeater.cc:511] Master 127.3.247.190:38343 requested a full tablet report, sending...
I20260806 06:51:35.369733 13289 ts_manager.cc:194] Registered new tserver with Master: bc091c207cb14a4b918ab2f586aa6aba (127.3.247.131:35475)
I20260806 06:51:35.370114 13289 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.131:46687
I20260806 06:51:35.377744 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:51:35.379514 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:35.379118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.3.247.190:38343)
0806 06:51:35.379166 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.379167 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.379257 (+ 90us) server_negotiation.cc:407] Connection header received
0806 06:51:35.379301 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.379302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.379311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.379358 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.379470 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58248: BlockingRecv error: recv got EOF from 127.0.0.1:58248 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:51:35.379585 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13349
I20260806 06:51:35.385183 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13480
I20260806 06:51:35.389086 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13611
I20260806 06:51:35.393409 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13258
2026-08-06T06:51:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (607 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T06:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:35Z Disabled control of system clock
I20260806 06:51:35.419318 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:42949
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:43245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:42949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:51:35.499140 13750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.499346 13750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.499368 13750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.500424 13750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:35.500456 13750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.500469 13750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.500481 13750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:35.500494 13750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:35.501742 13750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:42949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:42949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.502063 13750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.502367 13750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.507090 13755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.507102 13758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.507102 13756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.507447 13750 server_base.cc:1034] running on GCE node
I20260806 06:51:35.507685 13750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.507970 13750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.509133 13750 hybrid_clock.cc:648] HybridClock initialized: now 1785999095509126 us; error 35 us; skew 500 ppm
I20260806 06:51:35.510363 13750 webserver.cc:460] Webserver started at http://127.3.247.190:41659/ using document root <none> and password file <none>
I20260806 06:51:35.510613 13750 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.510684 13750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.510844 13750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.511967 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "6de6a017fc8b4151b8fdac727ef61c11"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "823c6754a0fe828a3ade6399117c92a5"
server_key_iv: "dd11a893b15a6bc4313f5e2169bb643c"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.512315 13750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "6de6a017fc8b4151b8fdac727ef61c11"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "823c6754a0fe828a3ade6399117c92a5"
server_key_iv: "dd11a893b15a6bc4313f5e2169bb643c"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.513680 13750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 06:51:35.514564 13764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.514724 13750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:35.514833 13750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "6de6a017fc8b4151b8fdac727ef61c11"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "823c6754a0fe828a3ade6399117c92a5"
server_key_iv: "dd11a893b15a6bc4313f5e2169bb643c"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.514910 13750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.518409 13750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.518723 13750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.518867 13750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.523061 13750 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:42949
I20260806 06:51:35.523104 13816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:42949 every 8 connection(s)
I20260806 06:51:35.523437 13750 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:35.524129 13817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:35.526947 13817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11: Bootstrap starting.
I20260806 06:51:35.527539 13817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:35.527851 13817 log.cc:824] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:35.528539 13817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11: No bootstrap required, opened a new log
I20260806 06:51:35.530201 13817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } }
I20260806 06:51:35.530349 13817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:35.530373 13817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de6a017fc8b4151b8fdac727ef61c11, State: Initialized, Role: FOLLOWER
I20260806 06:51:35.530547 13817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } }
I20260806 06:51:35.530618 13817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:35.530660 13817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:35.530709 13817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:35.531234 13817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } }
I20260806 06:51:35.531353 13817 leader_election.cc:302] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6de6a017fc8b4151b8fdac727ef61c11; no voters:
I20260806 06:51:35.531533 13817 leader_election.cc:288] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:35.531598 13820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:35.531795 13820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [term 1 LEADER]: Becoming Leader. State: Replica: 6de6a017fc8b4151b8fdac727ef61c11, State: Running, Role: LEADER
I20260806 06:51:35.531807 13817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:35.531908 13820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } }
I20260806 06:51:35.532290 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6de6a017fc8b4151b8fdac727ef61c11. Latest consensus state: current_term: 1 leader_uuid: "6de6a017fc8b4151b8fdac727ef61c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } } }
I20260806 06:51:35.532465 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:35.532502 13821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6de6a017fc8b4151b8fdac727ef61c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6de6a017fc8b4151b8fdac727ef61c11" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 42949 } } }
I20260806 06:51:35.532590 13821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:35.532936 13829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:35.533243 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13750
I20260806 06:51:35.533355 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:35.533510 13829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:35.533556 4062 external_mini_cluster.cc:1428] Setting key a8164d7e8ad4a8a010f449b33b56b88f
I20260806 06:51:35.535233 13829 catalog_manager.cc:1347] Generated new cluster ID: cdae7d78690f49d0af6f29ed3c78e84b
I20260806 06:51:35.535280 13829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:35.538702 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.534084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.3.247.190:42949)
0806 06:51:35.534649 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.534655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.534703 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:51:35.534836 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.534844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.534895 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.535030 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.538131 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.538134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.538135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.538156 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.538190 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.538203 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.538271 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.538389 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.538526 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":78,"threads_started":1}
I20260806 06:51:35.555413 13829 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:35.555621 13829 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:35.557436 13829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6de6a017fc8b4151b8fdac727ef61c11: Generated new TSK 0
I20260806 06:51:35.557639 13829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:51:35.562288 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--builtin_ntp_servers=127.3.247.148:43245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:51:35.644596 13841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.644831 13841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.644867 13841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.646075 13841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.646136 13841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.646178 13841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:35.647586 13841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.647859 13841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.648157 13841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.653890 13846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.653908 13847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.653898 13849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.654268 13841 server_base.cc:1034] running on GCE node
I20260806 06:51:35.654489 13841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.654778 13841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.655948 13841 hybrid_clock.cc:648] HybridClock initialized: now 1785999095655937 us; error 40 us; skew 500 ppm
I20260806 06:51:35.657339 13841 webserver.cc:460] Webserver started at http://127.3.247.129:33323/ using document root <none> and password file <none>
I20260806 06:51:35.657591 13841 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.657662 13841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.657879 13841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.658987 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "4b38314236864aa8975dcf87eeaad667"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "ef8ef661950277e47517bd2b0118600b"
server_key_iv: "ace5609e5352034da730ae7df4dc727f"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.659375 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "4b38314236864aa8975dcf87eeaad667"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "ef8ef661950277e47517bd2b0118600b"
server_key_iv: "ace5609e5352034da730ae7df4dc727f"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.660724 13841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:35.661471 13855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.661643 13841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:35.661744 13841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "4b38314236864aa8975dcf87eeaad667"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "ef8ef661950277e47517bd2b0118600b"
server_key_iv: "ace5609e5352034da730ae7df4dc727f"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.661862 13841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.665198 13841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.665470 13841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.665613 13841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.665867 13841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.666200 13841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.666254 13841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.666294 13841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.666321 13841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.672058 13841 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:45157
I20260806 06:51:35.672099 13968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:45157 every 8 connection(s)
I20260806 06:51:35.672410 13841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:35.675276 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.673226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:59523 (local address 127.3.247.190:42949)
0806 06:51:35.673315 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.673318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.673459 (+ 141us) server_negotiation.cc:407] Connection header received
0806 06:51:35.673932 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.673932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.673938 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.674000 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.675084 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.675086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.675086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.675089 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.675104 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.675108 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.675155 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.675211 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.675225 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 06:51:35.675483 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.673138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42949 (local address 127.3.247.129:59523)
0806 06:51:35.673328 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.673369 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.673848 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.674066 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.674070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.674077 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.674947 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.674947 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.674975 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.675003 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.675152 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.675154 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.675156 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.675206 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.675252 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260806 06:51:35.675947 13969 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42949
I20260806 06:51:35.676028 13969 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.676133 13969 heartbeater.cc:511] Master 127.3.247.190:42949 requested a full tablet report, sending...
I20260806 06:51:35.676563 13781 ts_manager.cc:194] Registered new tserver with Master: 4b38314236864aa8975dcf87eeaad667 (127.3.247.129:45157)
I20260806 06:51:35.677229 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13841
I20260806 06:51:35.677332 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:35.677513 4062 external_mini_cluster.cc:1428] Setting key c5a4dc4bbf285dce5f3d97012b324a21
I20260806 06:51:35.678431 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--builtin_ntp_servers=127.3.247.148:43245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:51:35.758417 13972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.758687 13972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.758736 13972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.760218 13972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.760294 13972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.760339 13972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:35.761708 13972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.762017 13972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.762354 13972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.767459 13977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.767545 13980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.767568 13972 server_base.cc:1034] running on GCE node
W20260806 06:51:35.767499 13978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.767875 13972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.768082 13972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.769218 13972 hybrid_clock.cc:648] HybridClock initialized: now 1785999095769194 us; error 42 us; skew 500 ppm
I20260806 06:51:35.770604 13972 webserver.cc:460] Webserver started at http://127.3.247.130:38009/ using document root <none> and password file <none>
I20260806 06:51:35.770843 13972 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.770890 13972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.770980 13972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.772013 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "55f4a07f8ba84f53a08d6c4b765cb317"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "81e02784bf504c1c33345efbcd872865"
server_key_iv: "0e61ad4b06f566c920cb880536781f56"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.772321 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "55f4a07f8ba84f53a08d6c4b765cb317"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "81e02784bf504c1c33345efbcd872865"
server_key_iv: "0e61ad4b06f566c920cb880536781f56"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.773561 13972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:35.774313 13986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.774497 13972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.774569 13972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "55f4a07f8ba84f53a08d6c4b765cb317"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "81e02784bf504c1c33345efbcd872865"
server_key_iv: "0e61ad4b06f566c920cb880536781f56"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.774621 13972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.777853 13972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.778072 13972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.778164 13972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.778321 13972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.778556 13972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.778585 13972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.778605 13972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.778618 13972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.784549 13972 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:39143
I20260806 06:51:35.784588 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:39143 every 8 connection(s)
I20260806 06:51:35.784907 13972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:51:35.788009 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.785641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:38495 (local address 127.3.247.190:42949)
0806 06:51:35.785705 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.785707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.785975 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:51:35.786539 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.786540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.786546 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.786577 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.787777 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.787779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.787779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.787784 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.787801 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.787805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.787860 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.787940 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.787953 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:51:35.788231 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.785647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42949 (local address 127.3.247.130:38495)
0806 06:51:35.785848 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.785894 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.786395 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.786740 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.786743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.786750 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.787631 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.787632 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.787660 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.787689 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.787868 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.787872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.787875 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.787934 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.787983 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260806 06:51:35.788770 14100 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42949
I20260806 06:51:35.788872 14100 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.789017 14100 heartbeater.cc:511] Master 127.3.247.190:42949 requested a full tablet report, sending...
I20260806 06:51:35.789412 13781 ts_manager.cc:194] Registered new tserver with Master: 55f4a07f8ba84f53a08d6c4b765cb317 (127.3.247.130:39143)
I20260806 06:51:35.793063 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 13972
I20260806 06:51:35.793147 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:35.793303 4062 external_mini_cluster.cc:1428] Setting key abca0dae957a6636191e74d1e7ad024f
I20260806 06:51:35.794243 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--builtin_ntp_servers=127.3.247.148:43245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:51:35.873840 14103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:35.874068 14103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:35.874089 14103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:35.875268 14103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:35.875308 14103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:35.875334 14103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:35.876657 14103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:35.876902 14103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:35.877192 14103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:35.882095 14109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:35.882242 14108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.882320 14103 server_base.cc:1034] running on GCE node
W20260806 06:51:35.882091 14111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:35.882566 14103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:35.882817 14103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:35.883957 14103 hybrid_clock.cc:648] HybridClock initialized: now 1785999095883944 us; error 30 us; skew 500 ppm
I20260806 06:51:35.885236 14103 webserver.cc:460] Webserver started at http://127.3.247.131:40003/ using document root <none> and password file <none>
I20260806 06:51:35.885480 14103 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:35.885526 14103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:35.885699 14103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:35.886893 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "1a198e9541e84bd08217daec508b58ee"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "1ab3fb4f7010a72a76d642753f881344"
server_key_iv: "be9d2fb7ef108a480b26f22a059986c6"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.887243 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "1a198e9541e84bd08217daec508b58ee"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "1ab3fb4f7010a72a76d642753f881344"
server_key_iv: "be9d2fb7ef108a480b26f22a059986c6"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.888666 14103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:51:35.889387 14117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.889596 14103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:35.889667 14103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "1a198e9541e84bd08217daec508b58ee"
format_stamp: "Formatted at 2026-08-06 06:51:35 on dist-test-slave-0kls"
server_key: "1ab3fb4f7010a72a76d642753f881344"
server_key_iv: "be9d2fb7ef108a480b26f22a059986c6"
server_key_version: "encryptionkey@0"
I20260806 06:51:35.889755 14103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:35.893041 14103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:35.893296 14103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:35.893431 14103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:35.893627 14103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:35.893985 14103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:35.894024 14103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.894044 14103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:35.894089 14103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:35.900033 14103 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:41965
I20260806 06:51:35.900087 14230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:41965 every 8 connection(s)
I20260806 06:51:35.900413 14103 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:35.903277 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.901145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:44013 (local address 127.3.247.190:42949)
0806 06:51:35.901202 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.901204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.901453 (+ 249us) server_negotiation.cc:407] Connection header received
0806 06:51:35.901931 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.901932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.901937 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.902005 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.903069 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:35.903071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:35.903071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:51:35.903075 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:35.903090 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:51:35.903095 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:35.903152 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:35.903213 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:51:35.903226 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:51:35.903477 14233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:35.901118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:42949 (local address 127.3.247.131:44013)
0806 06:51:35.901299 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:35.901339 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:35.901778 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:35.902058 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:35.902061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:35.902069 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:35.902928 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:51:35.902929 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:35.902957 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:51:35.902984 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:35.903151 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:35.903153 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:35.903155 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:51:35.903205 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:51:35.903249 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260806 06:51:35.904093 14231 heartbeater.cc:347] Connected to a master server at 127.3.247.190:42949
I20260806 06:51:35.904193 14231 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:35.904351 14231 heartbeater.cc:511] Master 127.3.247.190:42949 requested a full tablet report, sending...
I20260806 06:51:35.904726 13781 ts_manager.cc:194] Registered new tserver with Master: 1a198e9541e84bd08217daec508b58ee (127.3.247.131:41965)
I20260806 06:51:35.908762 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 14103
I20260806 06:51:35.908864 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:35.909039 4062 external_mini_cluster.cc:1428] Setting key 3099d1655a3a8d005cfc685f15a2396e
I20260806 06:51:35.909914 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:51:35.911623 13839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:51:35.911238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.3.247.190:42949)
0806 06:51:35.911293 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:35.911294 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:35.911400 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:51:35.911450 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:35.911451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:35.911455 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:35.911492 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:35.911578 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49846: BlockingRecv error: recv got EOF from 127.0.0.1:49846 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:51:35.911773 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13841
I20260806 06:51:35.915956 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13972
I20260806 06:51:35.920002 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 14103
I20260806 06:51:35.924840 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 13750
2026-08-06T06:51:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (531 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:35 dist-test-slave-0kls krb5kdc[14241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:35 dist-test-slave-0kls krb5kdc[14241](info): set up 2 sockets
Aug 06 06:51:35 dist-test-slave-0kls krb5kdc[14241](info): commencing operation
krb5kdc: starting...
W20260806 06:51:36.966815 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:51:37.083992 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39451
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:37.164320 14254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:37.164564 14254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:37.164604 14254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:37.165766 14254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:51:37.165880 14254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:37.165907 14254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:37.165936 14254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:51:37.165969 14254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:51:37.167454 14254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:37.167722 14254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:37.167946 14254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:37.172771 14259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.172942 14262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.172842 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.173012 14254 server_base.cc:1034] running on GCE node
I20260806 06:51:37.173291 14254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:37.173632 14254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:37.174773 14254 hybrid_clock.cc:648] HybridClock initialized: now 1785999097174746 us; error 45 us; skew 500 ppm
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:37.177230 14254 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:51:37.177712 14254 webserver.cc:460] Webserver started at http://127.3.247.190:45393/ using document root <none> and password file <none>
I20260806 06:51:37.177964 14254 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:37.178035 14254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:37.178185 14254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:37.179242 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "0678be0343564e2e98c5a74309c98e17"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "429f9d4e2d3ae4e17cc4cfcf031255a1"
server_key_iv: "ccc5a979bdcf788e4cb88850f3260ca9"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.179611 14254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "0678be0343564e2e98c5a74309c98e17"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "429f9d4e2d3ae4e17cc4cfcf031255a1"
server_key_iv: "ccc5a979bdcf788e4cb88850f3260ca9"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.180904 14254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:37.181600 14269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.181774 14254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:37.181922 14254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "0678be0343564e2e98c5a74309c98e17"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "429f9d4e2d3ae4e17cc4cfcf031255a1"
server_key_iv: "ccc5a979bdcf788e4cb88850f3260ca9"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.182010 14254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:37.190969 14254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:37.192711 14254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:37.192886 14254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:37.196904 14254 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39451
I20260806 06:51:37.196935 14321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39451 every 8 connection(s)
I20260806 06:51:37.197266 14254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:51:37.198310 14322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:37.198379 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 14254
I20260806 06:51:37.198484 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:51:37.198688 4062 external_mini_cluster.cc:1428] Setting key 68b5b7640710cecb56eee5e529387f8b
I20260806 06:51:37.201295 14322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: Bootstrap starting.
I20260806 06:51:37.201960 14322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:37.202237 14322 log.cc:824] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:37.202905 14322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: No bootstrap required, opened a new log
I20260806 06:51:37.204880 14322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } }
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:37.205050 14322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:37.205086 14322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0678be0343564e2e98c5a74309c98e17, State: Initialized, Role: FOLLOWER
I20260806 06:51:37.205225 14322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } }
I20260806 06:51:37.205344 14322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:51:37.205379 14322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:51:37.205430 14322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:37.206045 14322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } }
I20260806 06:51:37.206164 14322 leader_election.cc:302] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0678be0343564e2e98c5a74309c98e17; no voters:
I20260806 06:51:37.206338 14322 leader_election.cc:288] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:51:37.206395 14327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:37.206561 14327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [term 1 LEADER]: Becoming Leader. State: Replica: 0678be0343564e2e98c5a74309c98e17, State: Running, Role: LEADER
I20260806 06:51:37.206593 14322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:51:37.206653 14327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } }
I20260806 06:51:37.207024 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0678be0343564e2e98c5a74309c98e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } } }
I20260806 06:51:37.207015 14327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0678be0343564e2e98c5a74309c98e17. Latest consensus state: current_term: 1 leader_uuid: "0678be0343564e2e98c5a74309c98e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678be0343564e2e98c5a74309c98e17" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39451 } } }
I20260806 06:51:37.207190 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:37.207221 14327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17 [sys.catalog]: This master's current role is: LEADER
I20260806 06:51:37.207449 14336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:51:37.207999 14336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:51:37.209582 14336 catalog_manager.cc:1347] Generated new cluster ID: 7bc8401feddb4ab9a7b3d0feba9942e3
I20260806 06:51:37.209630 14336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:51:37.209870 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.199234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.3.247.190:39451)
0806 06:51:37.199417 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.199427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.199461 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:51:37.199900 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.199907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.199955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.200104 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:37.200721 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.201654 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.202537 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.202935 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.203057 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.205600 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:37.205604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:37.205604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:37.205613 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:37.207967 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.208377 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.208380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.208381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.208477 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.208720 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.208722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.208722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.209040 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:37.209165 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.209394 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.209641 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260806 06:51:37.230449 14336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:51:37.231185 14336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:51:37.241254 14336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: Generated new TSK 0
I20260806 06:51:37.241448 14336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:51:37.268853 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:37.348429 14348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:37.348663 14348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:37.348701 14348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:37.349874 14348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:37.349942 14348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:37.349987 14348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:51:37.351426 14348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:37.351678 14348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:37.351907 14348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:37.356902 14353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.356902 14356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.356946 14348 server_base.cc:1034] running on GCE node
W20260806 06:51:37.357125 14354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.357335 14348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:37.357599 14348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:37.358760 14348 hybrid_clock.cc:648] HybridClock initialized: now 1785999097358746 us; error 44 us; skew 500 ppm
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:37.361181 14348 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:51:37.361600 14348 webserver.cc:460] Webserver started at http://127.3.247.129:35307/ using document root <none> and password file <none>
I20260806 06:51:37.361896 14348 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:37.361964 14348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:37.362097 14348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:37.363130 14348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "ae854d4f7dcb41ee9c8e016f11179f46"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "87a3b88686fdb734945b9b98e5c79a57"
server_key_iv: "212b6fb6ecef71153cb7eeb7a74f109c"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.363513 14348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "ae854d4f7dcb41ee9c8e016f11179f46"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "87a3b88686fdb734945b9b98e5c79a57"
server_key_iv: "212b6fb6ecef71153cb7eeb7a74f109c"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.364813 14348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:51:37.365484 14363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.365641 14348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:51:37.365744 14348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "ae854d4f7dcb41ee9c8e016f11179f46"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "87a3b88686fdb734945b9b98e5c79a57"
server_key_iv: "212b6fb6ecef71153cb7eeb7a74f109c"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.365870 14348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:37.374549 14348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:37.376325 14348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:37.376480 14348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:37.376699 14348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:37.377002 14348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:37.377055 14348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.377096 14348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:37.377122 14348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.382925 14348 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:39233
I20260806 06:51:37.382968 14476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:39233 every 8 connection(s)
I20260806 06:51:37.383275 14348 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:51:37.383864 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 14348
I20260806 06:51:37.383975 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:51:37.384169 4062 external_mini_cluster.cc:1428] Setting key ad8992acacd79d1ebe71b1b2cfedb07d
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:51:37.394328 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.384395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:46125 (local address 127.3.247.190:39451)
0806 06:51:37.384471 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.384473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.384484 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:51:37.384922 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.384923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.384932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.384972 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:37.386343 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.387153 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.388447 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.388751 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.388841 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.391515 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:37.391516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:37.391517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:37.391524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:37.392870 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.393327 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.393328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.393329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.393387 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.393643 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.393644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.393644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.393766 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:37.393869 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.394135 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.394222 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:51:37.394505 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.383993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39451 (local address 127.3.247.129:46125)
0806 06:51:37.384158 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.384202 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.384803 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.385051 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.385056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.385953 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:37.386236 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.386244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.387256 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.387256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.388303 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.388306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.388852 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.388853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.389014 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.389595 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:37.389606 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:37.391407 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:37.392969 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.392976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.392978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.393247 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.393462 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.393463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.393464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.393576 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.393858 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:37.393865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:37.394004 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.394113 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.394267 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260806 06:51:37.394974 14477 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39451
I20260806 06:51:37.395080 14477 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:37.395263 14477 heartbeater.cc:511] Master 127.3.247.190:39451 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
I20260806 06:51:37.395850 14286 ts_manager.cc:194] Registered new tserver with Master: ae854d4f7dcb41ee9c8e016f11179f46 (127.3.247.129:39233)
I20260806 06:51:37.396791 14286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:46125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:51:37.408641 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:37.488322 14482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:37.488564 14482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:37.488605 14482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:37.489835 14482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:37.489904 14482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:37.489950 14482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:51:37.491555 14482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:37.491830 14482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:37.492062 14482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:37.496989 14488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.496948 14490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.496940 14487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.497133 14482 server_base.cc:1034] running on GCE node
I20260806 06:51:37.497380 14482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:37.497632 14482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:37.498798 14482 hybrid_clock.cc:648] HybridClock initialized: now 1785999097498778 us; error 53 us; skew 500 ppm
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:37.501195 14482 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:51:37.501641 14482 webserver.cc:460] Webserver started at http://127.3.247.130:42707/ using document root <none> and password file <none>
I20260806 06:51:37.501879 14482 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:37.501925 14482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:37.502014 14482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:37.503037 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "22fa479f93444de197af9980f2eebd0a"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "49e7be8bf65799e8e89c55afa46b0123"
server_key_iv: "83075a9d04dcbdb7fe47bde867eb13e1"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.503328 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "22fa479f93444de197af9980f2eebd0a"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "49e7be8bf65799e8e89c55afa46b0123"
server_key_iv: "83075a9d04dcbdb7fe47bde867eb13e1"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.504534 14482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:37.505178 14497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.505329 14482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:37.505411 14482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "22fa479f93444de197af9980f2eebd0a"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "49e7be8bf65799e8e89c55afa46b0123"
server_key_iv: "83075a9d04dcbdb7fe47bde867eb13e1"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.505465 14482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:37.517426 14482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:37.519755 14482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:37.519882 14482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:37.520069 14482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:37.520395 14482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:37.520433 14482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.520478 14482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:37.520504 14482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.526175 14482 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44727
I20260806 06:51:37.526252 14610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44727 every 8 connection(s)
I20260806 06:51:37.526577 14482 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:37.533160 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 14482
I20260806 06:51:37.533274 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:51:37.533459 4062 external_mini_cluster.cc:1428] Setting key 63cd94a1dc7db3c2c2b67f858e412b09
I20260806 06:51:37.536453 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.527366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:59033 (local address 127.3.247.190:39451)
0806 06:51:37.527437 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.527439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.527677 (+ 238us) server_negotiation.cc:407] Connection header received
0806 06:51:37.528216 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.528217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.528229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.528276 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:37.529266 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.529767 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.530791 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.530987 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.531070 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.533226 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:37.533227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:37.533228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:37.533237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:37.534866 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.535373 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.535374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.535375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.535433 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.535730 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.535731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.535732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.535893 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:37.535979 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.536224 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.536323 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:51:37.536466 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.527359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39451 (local address 127.3.247.130:59033)
0806 06:51:37.527511 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.527548 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.528089 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.528355 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.528359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.528931 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:37.529121 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.529125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.529889 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.529890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.530692 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.530693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.531078 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.531079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.531204 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.531676 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:37.531685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:37.533076 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:37.534952 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.534958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.534960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.535286 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.535500 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.535501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.535502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.535650 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.535970 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:37.535975 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:37.536091 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.536176 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.536295 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260806 06:51:37.536981 14611 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39451
I20260806 06:51:37.537089 14611 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:37.537288 14611 heartbeater.cc:511] Master 127.3.247.190:39451 requested a full tablet report, sending...
I20260806 06:51:37.537698 14286 ts_manager.cc:194] Registered new tserver with Master: 22fa479f93444de197af9980f2eebd0a (127.3.247.130:44727)
I20260806 06:51:37.538142 14286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:59033
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:51:37.560437 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:51:37.640620 14616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:51:37.640830 14616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:51:37.640854 14616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:51:37.642050 14616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:51:37.642091 14616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:51:37.642118 14616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:51:37.643554 14616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:51:37.643792 14616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:51:37.643972 14616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:51:37.648936 14622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:51:37.648928 14621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.649066 14616 server_base.cc:1034] running on GCE node
W20260806 06:51:37.648929 14624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:51:37.649386 14616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:51:37.649580 14616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:51:37.650712 14616 hybrid_clock.cc:648] HybridClock initialized: now 1785999097650691 us; error 38 us; skew 500 ppm
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:51:37.653294 14616 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:51:37.653721 14616 webserver.cc:460] Webserver started at http://127.3.247.131:46551/ using document root <none> and password file <none>
I20260806 06:51:37.653980 14616 fs_manager.cc:359] Metadata directory not provided
I20260806 06:51:37.654050 14616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:51:37.654203 14616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:51:37.655313 14616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "a1391b57645c4e35a403efdf6b3b7122"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "790581ff86f7166306199fb53c37d29d"
server_key_iv: "1818f2ea444aa0405b606ecac65dac51"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.655658 14616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "a1391b57645c4e35a403efdf6b3b7122"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "790581ff86f7166306199fb53c37d29d"
server_key_iv: "1818f2ea444aa0405b606ecac65dac51"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.656934 14616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:51:37.657644 14631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.657861 14616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:51:37.657941 14616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "a1391b57645c4e35a403efdf6b3b7122"
format_stamp: "Formatted at 2026-08-06 06:51:37 on dist-test-slave-0kls"
server_key: "790581ff86f7166306199fb53c37d29d"
server_key_iv: "1818f2ea444aa0405b606ecac65dac51"
server_key_version: "encryptionkey@0"
I20260806 06:51:37.658034 14616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:51:37.672457 14616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:51:37.674917 14616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:51:37.675077 14616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:51:37.675325 14616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:51:37.675642 14616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:51:37.675676 14616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.675699 14616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:51:37.675712 14616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:51:37.681257 14616 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:39267
I20260806 06:51:37.681360 14744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:39267 every 8 connection(s)
I20260806 06:51:37.681664 14616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:51:37.685719 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 14616
I20260806 06:51:37.685845 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:51:37.686064 4062 external_mini_cluster.cc:1428] Setting key 532fabd5acdd3c492c33b59f161df8b7
Aug 06 06:51:37 dist-test-slave-0kls krb5kdc[14241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999097, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:51:37.691181 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.682475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:51735 (local address 127.3.247.190:39451)
0806 06:51:37.682546 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.682547 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.682722 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:51:37.683189 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.683190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.683214 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.683277 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:37.684211 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.684752 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.685996 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.686264 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.686334 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.688541 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:37.688543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:37.688543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:37.688549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:37.690003 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.690342 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.690343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.690343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.690394 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.690592 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.690592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.690593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.690689 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:37.690741 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.691046 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.691118 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:51:37.691305 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.682452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39451 (local address 127.3.247.131:51735)
0806 06:51:37.682592 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.682634 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.683100 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.683330 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.683333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.683886 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:51:37.684073 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.684077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.684826 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.684827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.685872 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.685876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.686412 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.686413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.686546 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.687130 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:51:37.687137 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:51:37.688460 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:51:37.690066 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.690071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.690072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.690279 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.690449 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:51:37.690450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:51:37.690450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:51:37.690539 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:51:37.690743 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:51:37.690747 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:51:37.690901 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.691005 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.691122 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260806 06:51:37.691759 14745 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39451
I20260806 06:51:37.691836 14745 heartbeater.cc:464] Registering TS with master...
I20260806 06:51:37.692019 14745 heartbeater.cc:511] Master 127.3.247.190:39451 requested a full tablet report, sending...
I20260806 06:51:37.692380 14286 ts_manager.cc:194] Registered new tserver with Master: a1391b57645c4e35a403efdf6b3b7122 (127.3.247.131:39267)
I20260806 06:51:37.692718 14286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:51735
I20260806 06:51:37.697966 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:51:37.704900 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.699289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.3.247.190:39451)
0806 06:51:37.699359 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.699361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.699398 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:51:37.699446 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.699447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.699455 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.699506 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:51:37.699985 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.700488 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.701216 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.701385 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.701433 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.702631 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:51:37.702632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:51:37.702633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:51:37.702638 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:51:37.703990 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.704279 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.704280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.704280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.704327 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:51:37.704461 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:51:37.704461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:51:37.704461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:51:37.704595 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:51:37.704669 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.704755 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.704808 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:51:37.706440 14286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:51:37.708045 14286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:51:37.714347 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44727 (local address 127.0.0.1:41838)
0806 06:51:37.710717 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.710725 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.710789 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.711191 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.711192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.711193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:37.711225 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.711226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.712028 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.712029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.713165 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.713168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.714038 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.714112 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.714127 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.714205 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.714266 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":49,"thread_start_us":31,"threads_started":1}
I20260806 06:51:37.714375 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.3.247.130:44727)
0806 06:51:37.710978 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.710982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.710991 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:37.711027 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.711029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.711042 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.711108 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:37.711276 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.711921 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.713253 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.713952 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714029 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.714085 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.714239 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.714294 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":100,"threads_started":1}
I20260806 06:51:37.714588 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39267 (local address 127.0.0.1:55098)
0806 06:51:37.710454 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.710466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.710516 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.711119 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.711120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.711122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:37.711165 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.711166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.712339 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.712340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.713494 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.713495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.714347 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.714417 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.714421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.714478 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.714523 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":55,"threads_started":1}
I20260806 06:51:37.714728 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39233 (local address 127.0.0.1:60902)
0806 06:51:37.710290 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.710325 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.710386 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.710613 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.710615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.710634 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:37.710695 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.710697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.711978 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.711979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.713372 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.713374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.714303 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.714367 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.714371 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.714402 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.714645 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":77,"thread_start_us":60,"threads_started":1}
I20260806 06:51:37.714864 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.3.247.131:39267)
0806 06:51:37.710839 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.710844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.710853 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:37.710888 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.710890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.710904 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.711043 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:37.711240 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.712247 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.713577 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.714209 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714265 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.714315 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.714747 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.714785 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":102,"threads_started":1}
I20260806 06:51:37.715162 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.710286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.3.247.129:39233)
0806 06:51:37.710461 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.710466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.710475 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:37.710510 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.710512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.710531 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.710610 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:37.710788 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.711870 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.713458 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.714206 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.714961 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.715021 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.715046 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.715086 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260806 06:51:37.715274 14545 tablet_service.cc:1467] Processing CreateTablet for tablet 29574e29486045b9b9191363353c6ff6 (DEFAULT_TABLE table=test-table [id=2ad505ab633f4234bba4cefbc42ed724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:37.715631 14545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29574e29486045b9b9191363353c6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:37.715875 14411 tablet_service.cc:1467] Processing CreateTablet for tablet 29574e29486045b9b9191363353c6ff6 (DEFAULT_TABLE table=test-table [id=2ad505ab633f4234bba4cefbc42ed724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:37.716177 14411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29574e29486045b9b9191363353c6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:37.718448 14679 tablet_service.cc:1467] Processing CreateTablet for tablet 29574e29486045b9b9191363353c6ff6 (DEFAULT_TABLE table=test-table [id=2ad505ab633f4234bba4cefbc42ed724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:51:37.718788 14679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29574e29486045b9b9191363353c6ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:51:37.719317 14760 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Bootstrap starting.
I20260806 06:51:37.720144 14760 tablet_bootstrap.cc:654] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:37.720779 14760 log.cc:824] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:37.721643 14760 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: No bootstrap required, opened a new log
I20260806 06:51:37.721730 14760 ts_tablet_manager.cc:1397] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:51:37.722726 14762 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Bootstrap starting.
I20260806 06:51:37.722832 14759 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Bootstrap starting.
I20260806 06:51:37.723338 14762 tablet_bootstrap.cc:654] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:37.723647 14759 tablet_bootstrap.cc:654] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Neither blocks nor log segments found. Creating new log.
I20260806 06:51:37.723842 14762 log.cc:824] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:37.724261 14759 log.cc:824] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:51:37.724436 14760 raft_consensus.cc:348] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.724620 14762 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: No bootstrap required, opened a new log
I20260806 06:51:37.724655 14760 raft_consensus.cc:374] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:37.724682 14762 ts_tablet_manager.cc:1397] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:51:37.724735 14760 raft_consensus.cc:729] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae854d4f7dcb41ee9c8e016f11179f46, State: Initialized, Role: FOLLOWER
I20260806 06:51:37.724934 14760 consensus_queue.cc:260] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.725096 14759 tablet_bootstrap.cc:492] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: No bootstrap required, opened a new log
I20260806 06:51:37.725214 14759 ts_tablet_manager.cc:1397] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:51:37.725317 14477 heartbeater.cc:503] Master 127.3.247.190:39451 was elected leader, sending a full tablet report...
I20260806 06:51:37.725641 14760 ts_tablet_manager.cc:1428] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:51:37.726634 14762 raft_consensus.cc:348] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.726783 14762 raft_consensus.cc:374] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:37.726807 14762 raft_consensus.cc:729] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1391b57645c4e35a403efdf6b3b7122, State: Initialized, Role: FOLLOWER
I20260806 06:51:37.726946 14762 consensus_queue.cc:260] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.727257 14762 ts_tablet_manager.cc:1428] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:51:37.727286 14745 heartbeater.cc:503] Master 127.3.247.190:39451 was elected leader, sending a full tablet report...
I20260806 06:51:37.727782 14759 raft_consensus.cc:348] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.727980 14759 raft_consensus.cc:374] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:51:37.728042 14759 raft_consensus.cc:729] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fa479f93444de197af9980f2eebd0a, State: Initialized, Role: FOLLOWER
I20260806 06:51:37.728180 14759 consensus_queue.cc:260] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.728516 14759 ts_tablet_manager.cc:1428] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:51:37.728526 14611 heartbeater.cc:503] Master 127.3.247.190:39451 was elected leader, sending a full tablet report...
W20260806 06:51:37.777120 14612 tablet.cc:2369] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:51:37.884207 14478 tablet.cc:2369] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:51:37.932274 14746 tablet.cc:2369] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:51:37.938963 14767 raft_consensus.cc:482] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:51:37.939131 14767 raft_consensus.cc:504] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.939590 14767 leader_election.cc:288] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1391b57645c4e35a403efdf6b3b7122 (127.3.247.131:39267), ae854d4f7dcb41ee9c8e016f11179f46 (127.3.247.129:39233)
I20260806 06:51:37.942445 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.939778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39233 (local address 127.3.247.130:60557)
0806 06:51:37.939946 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.939957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.940015 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.940154 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.940154 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.940156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:37.940238 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.940240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.940861 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.940862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.941602 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.941604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.942190 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.942191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.942273 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.942277 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.942319 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.942350 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260806 06:51:37.942463 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.939844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:60557 (local address 127.3.247.129:39233)
0806 06:51:37.939910 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.939912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.940036 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:51:37.940064 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.940065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.940075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.940134 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:37.940322 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.940779 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.941727 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.942108 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.942163 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.942170 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.942343 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.942392 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:51:37.942545 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.939853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:53069 (local address 127.3.247.131:39267)
0806 06:51:37.939907 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.939910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.939945 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:51:37.939974 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.939975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.939985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.940038 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:51:37.940229 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.940718 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.941687 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.942130 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.942217 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.942224 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.942432 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.942478 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:51:37.942591 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.939753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:39267 (local address 127.3.247.130:53069)
0806 06:51:37.939831 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:37.939851 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 06:51:37.939899 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:51:37.940058 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:51:37.940060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:51:37.940061 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:51:37.940147 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.940148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.940801 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.940802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.941586 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:51:37.941588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.942243 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.942243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:51:37.942346 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.942350 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:51:37.942413 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:51:37.942451 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":25}
I20260806 06:51:37.943037 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29574e29486045b9b9191363353c6ff6" candidate_uuid: "22fa479f93444de197af9980f2eebd0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" is_pre_election: true
I20260806 06:51:37.943042 14699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29574e29486045b9b9191363353c6ff6" candidate_uuid: "22fa479f93444de197af9980f2eebd0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1391b57645c4e35a403efdf6b3b7122" is_pre_election: true
I20260806 06:51:37.943212 14431 raft_consensus.cc:2454] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fa479f93444de197af9980f2eebd0a in term 0.
I20260806 06:51:37.943212 14699 raft_consensus.cc:2454] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fa479f93444de197af9980f2eebd0a in term 0.
I20260806 06:51:37.943424 14499 leader_election.cc:302] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fa479f93444de197af9980f2eebd0a, a1391b57645c4e35a403efdf6b3b7122; no voters:
I20260806 06:51:37.943569 14767 raft_consensus.cc:2781] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:51:37.943647 14767 raft_consensus.cc:482] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:51:37.943670 14767 raft_consensus.cc:3037] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:37.944428 14767 raft_consensus.cc:504] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.944581 14767 leader_election.cc:288] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [CANDIDATE]: Term 1 election: Requested vote from peers a1391b57645c4e35a403efdf6b3b7122 (127.3.247.131:39267), ae854d4f7dcb41ee9c8e016f11179f46 (127.3.247.129:39233)
I20260806 06:51:37.944734 14699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29574e29486045b9b9191363353c6ff6" candidate_uuid: "22fa479f93444de197af9980f2eebd0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1391b57645c4e35a403efdf6b3b7122"
I20260806 06:51:37.944749 14431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29574e29486045b9b9191363353c6ff6" candidate_uuid: "22fa479f93444de197af9980f2eebd0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae854d4f7dcb41ee9c8e016f11179f46"
I20260806 06:51:37.944823 14699 raft_consensus.cc:3037] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:37.944823 14431 raft_consensus.cc:3037] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:51:37.945698 14431 raft_consensus.cc:2454] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fa479f93444de197af9980f2eebd0a in term 1.
I20260806 06:51:37.945699 14699 raft_consensus.cc:2454] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fa479f93444de197af9980f2eebd0a in term 1.
I20260806 06:51:37.945909 14500 leader_election.cc:302] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fa479f93444de197af9980f2eebd0a, ae854d4f7dcb41ee9c8e016f11179f46; no voters:
I20260806 06:51:37.946010 14767 raft_consensus.cc:2781] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:51:37.946180 14767 raft_consensus.cc:686] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 1 LEADER]: Becoming Leader. State: Replica: 22fa479f93444de197af9980f2eebd0a, State: Running, Role: LEADER
I20260806 06:51:37.946285 14767 consensus_queue.cc:237] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } }
I20260806 06:51:37.947167 14286 catalog_manager.cc:5495] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22fa479f93444de197af9980f2eebd0a (127.3.247.130). New cstate: current_term: 1 leader_uuid: "22fa479f93444de197af9980f2eebd0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fa479f93444de197af9980f2eebd0a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44727 } health_report { overall_health: HEALTHY } } }
I20260806 06:51:37.997577 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:37.995302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.3.247.130:44727)
0806 06:51:37.995350 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:51:37.995352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:51:37.995361 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:51:37.995406 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:51:37.995407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:51:37.995424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:51:37.995491 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:51:37.995621 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.996104 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.996854 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:51:37.997059 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:51:37.997151 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:51:37.997266 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:51:37.997369 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:51:37.997431 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:51:37.997462 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:51:37.997483 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:51:37.999684 14699 raft_consensus.cc:1261] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 1 FOLLOWER]: Refusing update from remote peer 22fa479f93444de197af9980f2eebd0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:37.999725 14431 raft_consensus.cc:1261] T 29574e29486045b9b9191363353c6ff6 P ae854d4f7dcb41ee9c8e016f11179f46 [term 1 FOLLOWER]: Refusing update from remote peer 22fa479f93444de197af9980f2eebd0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:51:38.000077 14769 consensus_queue.cc:1035] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1391b57645c4e35a403efdf6b3b7122" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 39267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:38.000241 14767 consensus_queue.cc:1035] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae854d4f7dcb41ee9c8e016f11179f46" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:51:38.002753 14772 mvcc.cc:205] Tried to move back new op lower bound from 7315452305403469824 to 7315452305188630528. Current Snapshot: MvccSnapshot[applied={T|T < 7315452305403469824}]
I20260806 06:51:38.002931 14774 mvcc.cc:205] Tried to move back new op lower bound from 7315452305403469824 to 7315452305188630528. Current Snapshot: MvccSnapshot[applied={T|T < 7315452305403469824}]
I20260806 06:51:38.005381 14286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:38.005501 14286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:51:38.007110 14286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0678be0343564e2e98c5a74309c98e17: Sending DeleteTablet for 3 replicas of tablet 29574e29486045b9b9191363353c6ff6
I20260806 06:51:38.007442 14679 tablet_service.cc:1514] Processing DeleteTablet for tablet 29574e29486045b9b9191363353c6ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:55098
I20260806 06:51:38.007827 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 14348
I20260806 06:51:38.008028 14545 tablet_service.cc:1514] Processing DeleteTablet for tablet 29574e29486045b9b9191363353c6ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:41838
I20260806 06:51:38.008214 14544 tablet_service.cc:1514] Processing DeleteTablet for tablet 29574e29486045b9b9191363353c6ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:51:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:41838
I20260806 06:51:38.008490 14780 tablet_replica.cc:330] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: stopping tablet replica
I20260806 06:51:38.009999 14780 raft_consensus.cc:2229] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:51:38.010387 14780 raft_consensus.cc:2258] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:38.011341 14780 ts_tablet_manager.cc:1905] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:51:38.013464 14780 ts_tablet_manager.cc:1918] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:38.013552 14780 log.cc:1196] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/29574e29486045b9b9191363353c6ff6
I20260806 06:51:38.014026 14780 ts_tablet_manager.cc:1939] T 29574e29486045b9b9191363353c6ff6 P a1391b57645c4e35a403efdf6b3b7122: Deleting consensus metadata
I20260806 06:51:38.014060 14781 tablet_replica.cc:330] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: stopping tablet replica
I20260806 06:51:38.014185 14781 raft_consensus.cc:2229] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:51:38.014323 14781 raft_consensus.cc:2258] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:51:38.014744 14271 catalog_manager.cc:4841] TS a1391b57645c4e35a403efdf6b3b7122 (127.3.247.131:39267): tablet 29574e29486045b9b9191363353c6ff6 (table test-table [id=2ad505ab633f4234bba4cefbc42ed724]) successfully deleted
I20260806 06:51:38.014798 14781 ts_tablet_manager.cc:1905] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:51:38.016350 14272 connection.cc:535] client connection to 127.3.247.129:39233 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:51:38.016362 14781 ts_tablet_manager.cc:1918] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:51:38.016427 14781 log.cc:1196] T 29574e29486045b9b9191363353c6ff6 P 22fa479f93444de197af9980f2eebd0a: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/29574e29486045b9b9191363353c6ff6
W20260806 06:51:38.016448 14272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:51:38.016598 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 14482
I20260806 06:51:38.017952 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:38.017766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:39233 (local address 127.0.0.1:60916)
0806 06:51:38.017870 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:38.017921 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260806 06:51:38.018148 14272 catalog_manager.cc:4584] TS ae854d4f7dcb41ee9c8e016f11179f46 (127.3.247.129:39233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29574e29486045b9b9191363353c6ff6: Network error: Client connection negotiation failed: client connection to 127.3.247.129:39233: connect: Connection refused (error 111)
I20260806 06:51:38.021894 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 14616
I20260806 06:51:38.021941 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:51:38.021825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44727 (local address 127.0.0.1:41844)
0806 06:51:38.021875 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:51:38.021907 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.130:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:51:38.026053 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 14254
2026-08-06T06:51:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2102 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:51:38 dist-test-slave-0kls krb5kdc[14786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:51:38 dist-test-slave-0kls krb5kdc[14786](info): set up 2 sockets
Aug 06 06:51:38 dist-test-slave-0kls krb5kdc[14786](info): commencing operation
krb5kdc: starting...
W20260806 06:51:39.090955 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:51:39 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:51:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangeradmin_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangeradmin_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangerlookup_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangerlookup_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
I20260806 06:51:39.253342 4062 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktauUQm/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/postgres -l logfile start
2026-08-06 06:51:41.739 UTC [14816] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:51:41.739 UTC [14816] LOG: listening on IPv4 address "127.3.247.148", port 42637
2026-08-06 06:51:41.743 UTC [14816] LOG: listening on Unix socket "/tmp/.s.PGSQL.42637"
2026-08-06 06:51:41.763 UTC [14819] LOG: database system was shut down at 2026-08-06 06:51:40 UTC
2026-08-06 06:51:41.767 UTC [14816] LOG: database system is ready to accept connections
I20260806 06:51:42.718578 4062 mini_postgres.cc:96] Postgres bound to 42637
2026-08-06 06:51:42.724 UTC [14827] FATAL: database "slave" does not exist
127.3.247.148:42637 - accepting connections
I20260806 06:51:42.724932 4062 mini_ranger.cc:161] Starting Ranger...
I20260806 06:51:42.739140 4062 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:51:43.189028 4062 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:51:43.189105 4062 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin
2026-08-06 06:51:43,447 [I] DB FLAVOR :POSTGRES
2026-08-06 06:51:43,447 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:51:43,447 [I] Checking connection..
2026-08-06 06:51:43,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:51:43,682 [I] Checking connection passed.
2026-08-06 06:51:43,682 [I] --------- Verifying version history table ---------
2026-08-06 06:51:43,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:51:43,926 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:51:43,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:51:44,169 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:51:44,169 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:51:44,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:51:44.405 UTC [14891] WARNING: there is no transaction in progress
2026-08-06 06:51:44,411 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:51:44,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:51:44,669 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:51:44,670 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:51:44,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:51:44,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:51:45,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0kls.c.gcp-upstream.internal','N') ;"
2026-08-06 06:51:45,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:51:45,622 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:51:45,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:51:45,890 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:51:45,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:51:46,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:51:46,359 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:51:46,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:51:47.319 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.335 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.351 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.367 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.383 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.401 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.466 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.473 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.488 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.498 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.508 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.523 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.533 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.540 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.547 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:47.554 UTC [14999] WARNING: there is no transaction in progress
2026-08-06 06:51:48,186 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:51:48,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:51:48,409 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:51:48,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:51:48,641 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:51:48,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:51:48,908 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:51:48,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:51:49,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:51:49,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0kls.c.gcp-upstream.internal';"
2026-08-06 06:51:49,653 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:51:49,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:51:49,880 [I] DB_PATCHES have already been applied
I20260806 06:51:49.885533 4062 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktauUQm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:51:49.885592 4062 mini_ranger.cc:193] Using host: 127.3.247.148
I20260806 06:51:49.888480 4062 mini_ranger.cc:227] Ranger admin URL: http://127.3.247.148:55153
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:51:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:51:51.008315 4062 mini_ranger.cc:160] Time spent starting Ranger: real 8.283s user 0.003s sys 0.006s
Aug 06, 2026 6:52:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:52:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:52:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:52:10 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:52:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:52:11.993974 4062 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangerkms_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/rangerkms_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.247.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.247.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:52:12 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999132, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:52:12.082109 4062 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:52:12.091414 4062 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:52:12.871706 4062 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:52:12.871786 4062 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms
I20260806 06:52:12.871802 4062 mini_ranger_kms.cc:227] Using postgres at 127.3.247.148:42637
2026-08-06 06:52:13,064 [I] DB FLAVOR :POSTGRES
2026-08-06 06:52:13,065 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:52:13,065 [I] Checking connection
2026-08-06 06:52:13,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:52:13,332 [I] connection success
2026-08-06 06:52:13,333 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:52:13,333 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:52:13,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:52:13,612 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:52:13,612 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:52:13,612 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:52:13,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:42637/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:52:13,905 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:52:14.377278 4062 mini_ranger_kms.cc:310] Created kms service
I20260806 06:52:14.558171 4062 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:52:14.601051 4062 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:52:14 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999132, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:52:15.036734 4062 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:52:15.036854 4062 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:52:15.036876 4062 mini_ranger_kms.cc:240] Using host: 127.3.247.148
I20260806 06:52:15.040592 4062 mini_ranger_kms.cc:276] Ranger KMS PID: 15179
I20260806 06:52:15.040728 4062 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.247.148:41779
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:52:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:52:17.298089 4062 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.216s user 0.011s sys 0.164s
I20260806 06:52:17.298202 4062 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:52:17.298285 4062 mini_ranger_kms.cc:401] 127.3.247.148:41779/kms/v1/keys
Aug 06 06:52:17 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785999137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:52:17 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
Aug 06 06:52:18 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
Aug 06 06:52:20 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:52:20.553161 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:44613
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:44613
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:41779/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:52:20.645455 15230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:52:20.645705 15230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:52:20.645743 15230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:52:20.647019 15230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:52:20.647081 15230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:52:20.647106 15230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:52:20.647137 15230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:52:20.647164 15230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:52:20.648828 15230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.247.148:41779/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:44613
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:52:20.649118 15230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:52:20.649350 15230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:52:20.654695 15236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:20.654676 15235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:20.654663 15238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:20.654704 15230 server_base.cc:1034] running on GCE node
I20260806 06:52:20.655210 15230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:52:20.655542 15230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:52:20.656674 15230 hybrid_clock.cc:648] HybridClock initialized: now 1785999140656648 us; error 40 us; skew 500 ppm
Aug 06 06:52:20 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999140, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:52:20.664250 15230 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:52:20.664858 15230 webserver.cc:460] Webserver started at http://127.3.247.190:42631/ using document root <none> and password file <none>
I20260806 06:52:20.665114 15230 fs_manager.cc:359] Metadata directory not provided
I20260806 06:52:20.665177 15230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:52:20.665334 15230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:52:20 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999140, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:52:20.726019 15230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "26f0725bc3e24438a35a596600ba201b"
format_stamp: "Formatted at 2026-08-06 06:52:20 on dist-test-slave-0kls"
server_key: "0e4bc0f1af4547b81467e9c69f585d62"
server_key_iv: "664a6b093c354a1115476be4f1153986"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:20.726421 15230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "26f0725bc3e24438a35a596600ba201b"
format_stamp: "Formatted at 2026-08-06 06:52:20 on dist-test-slave-0kls"
server_key: "0e4bc0f1af4547b81467e9c69f585d62"
server_key_iv: "664a6b093c354a1115476be4f1153986"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:20.727818 15230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:52:20.743156 15245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:20.743350 15230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:52:20.743456 15230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "26f0725bc3e24438a35a596600ba201b"
format_stamp: "Formatted at 2026-08-06 06:52:20 on dist-test-slave-0kls"
server_key: "0e4bc0f1af4547b81467e9c69f585d62"
server_key_iv: "664a6b093c354a1115476be4f1153986"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:20.743569 15230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:52:20.771133 15230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:52:20.773129 15230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:52:20.773303 15230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:52:20.778651 15230 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:44613
I20260806 06:52:20.778683 15297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:44613 every 8 connection(s)
I20260806 06:52:20.779044 15230 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:52:20.779758 15298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:52:20.780157 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 15230
I20260806 06:52:20.780267 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:52:20.780467 4062 external_mini_cluster.cc:1428] Setting key 2461eadb856f6d923e4dc3ecb5727748
I20260806 06:52:20.782613 15298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: Bootstrap starting.
I20260806 06:52:20.783375 15298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: Neither blocks nor log segments found. Creating new log.
I20260806 06:52:20.783741 15298 log.cc:824] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:52:20.784428 15298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: No bootstrap required, opened a new log
I20260806 06:52:20.787113 15298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } }
I20260806 06:52:20.787335 15298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:52:20.787397 15298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f0725bc3e24438a35a596600ba201b, State: Initialized, Role: FOLLOWER
I20260806 06:52:20.787573 15298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } }
I20260806 06:52:20.787683 15298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:52:20.787719 15298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:52:20.787810 15298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:52:20.788606 15298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } }
I20260806 06:52:20.788730 15298 leader_election.cc:302] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26f0725bc3e24438a35a596600ba201b; no voters:
I20260806 06:52:20.788977 15298 leader_election.cc:288] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:52:20.789045 15303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:52:20.789304 15303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [term 1 LEADER]: Becoming Leader. State: Replica: 26f0725bc3e24438a35a596600ba201b, State: Running, Role: LEADER
I20260806 06:52:20.789412 15298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:52:20.789472 15303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } }
Aug 06 06:52:20 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:52:20.789902 15303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26f0725bc3e24438a35a596600ba201b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } } }
I20260806 06:52:20.790040 15303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [sys.catalog]: This master's current role is: LEADER
I20260806 06:52:20.790374 15312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:52:20.790999 15312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:52:20.791324 15298 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:52:20.795445 15319 mvcc.cc:205] Tried to move back new op lower bound from 7315452480690716672 to 7315452480674349056. Current Snapshot: MvccSnapshot[applied={T|T < 7315452480690716672}]
I20260806 06:52:20.797551 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:20.781023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.3.247.190:44613)
0806 06:52:20.781224 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:20.781235 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:20.782038 (+ 803us) server_negotiation.cc:407] Connection header received
0806 06:52:20.782623 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:20.782635 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:20.782703 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:20.782857 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:52:20.783736 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:20.785115 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:20.786304 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:20.786940 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:20.787060 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:20.790170 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:52:20.790174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:52:20.790174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:52:20.790187 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:52:20.794842 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:20.795429 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:20.795434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:20.795434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:20.795601 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:20.795854 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:20.795854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:20.795855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:20.796355 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:52:20.797012 (+ 657us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:20.797091 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:52:20.797272 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260806 06:52:20.790207 15304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26f0725bc3e24438a35a596600ba201b. Latest consensus state: current_term: 1 leader_uuid: "26f0725bc3e24438a35a596600ba201b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f0725bc3e24438a35a596600ba201b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44613 } } }
I20260806 06:52:20.799242 15304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b [sys.catalog]: This master's current role is: LEADER
I20260806 06:52:20.802356 15312 catalog_manager.cc:1347] Generated new cluster ID: b6ed5b10ef7c4ff580e590aa4a9f457e
I20260806 06:52:20.802418 15312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:52:20.829752 15312 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:52:20.830722 15312 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:52:20.836480 15312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: Generated new TSK 0
I20260806 06:52:20.836781 15312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:52:21.604532 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:41779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:52:21.693898 15343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:52:21.694188 15343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:52:21.694229 15343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:52:21.695453 15343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:52:21.695540 15343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:52:21.695621 15343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:52:21.697142 15343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.247.148:41779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:52:21.697419 15343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:52:21.697649 15343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:52:21.702714 15351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:21.702802 15343 server_base.cc:1034] running on GCE node
W20260806 06:52:21.702709 15349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:21.702860 15348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:21.703200 15343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:52:21.703483 15343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:52:21.704639 15343 hybrid_clock.cc:648] HybridClock initialized: now 1785999141704619 us; error 38 us; skew 500 ppm
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:52:21.707803 15343 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:52:21.708280 15343 webserver.cc:460] Webserver started at http://127.3.247.129:34203/ using document root <none> and password file <none>
I20260806 06:52:21.708518 15343 fs_manager.cc:359] Metadata directory not provided
I20260806 06:52:21.708588 15343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:52:21.708724 15343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:52:21.722515 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "6f4f48a80ed346ab9653c8b1ba564f43"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "e2bff01d7ff1368c4555ee093e39bd2c"
server_key_iv: "16dfa868a29d7729b776c3f00619260f"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.723070 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "6f4f48a80ed346ab9653c8b1ba564f43"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "e2bff01d7ff1368c4555ee093e39bd2c"
server_key_iv: "16dfa868a29d7729b776c3f00619260f"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.724629 15343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:52:21.737848 15358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:21.738149 15343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:52:21.738241 15343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "6f4f48a80ed346ab9653c8b1ba564f43"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "e2bff01d7ff1368c4555ee093e39bd2c"
server_key_iv: "16dfa868a29d7729b776c3f00619260f"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.738340 15343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:52:21.758682 15343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:52:21.761339 15343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:52:21.761494 15343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:52:21.761735 15343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:52:21.762141 15343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:52:21.762198 15343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:21.762230 15343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:52:21.762254 15343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:21.769878 15343 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:45855
I20260806 06:52:21.770318 15343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:52:21.771031 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 15343
I20260806 06:52:21.771144 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:52:21.771358 4062 external_mini_cluster.cc:1428] Setting key c895da3755db1ca66f7fc42314139706
I20260806 06:52:21.773940 15473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:45855 every 8 connection(s)
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:52:21.786605 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:21.771422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:54283 (local address 127.3.247.190:44613)
0806 06:52:21.775823 (+ 4401us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:21.775826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:21.775835 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:52:21.775860 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:21.775860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:21.775872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:21.775950 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:52:21.776758 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:21.777589 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:21.779916 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:21.780221 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:21.780289 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:21.782922 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:52:21.782923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:52:21.782924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:52:21.782932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:52:21.784850 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:21.785384 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:21.785386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:21.785386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:21.785451 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:21.785721 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:21.785722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:21.785723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:21.785910 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:52:21.785986 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:21.786414 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 06:52:21.786501 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4372,"thread_start_us":85,"threads_started":1}
I20260806 06:52:21.786790 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:21.771111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44613 (local address 127.3.247.129:54283)
0806 06:52:21.773726 (+ 2615us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:21.773778 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:21.774422 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:21.776072 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:21.776078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:21.776337 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:52:21.776637 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:21.776644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:21.777699 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:21.777700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:21.779808 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:21.779812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:21.780356 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:21.780357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:21.780508 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:21.781129 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:52:21.781139 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:52:21.782811 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:52:21.784960 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:21.784966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:21.784968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:21.785298 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:21.785521 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:21.785522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:21.785522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:21.785635 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:21.785978 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:52:21.785984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:52:21.786153 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 06:52:21.786381 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 06:52:21.786533 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2563,"thread_start_us":87,"threads_started":1}
I20260806 06:52:21.787355 15474 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44613
I20260806 06:52:21.787474 15474 heartbeater.cc:464] Registering TS with master...
I20260806 06:52:21.787663 15474 heartbeater.cc:511] Master 127.3.247.190:44613 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
I20260806 06:52:21.788205 15262 ts_manager.cc:194] Registered new tserver with Master: 6f4f48a80ed346ab9653c8b1ba564f43 (127.3.247.129:45855)
I20260806 06:52:21.788841 15262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:54283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:52:21.809755 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:41779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:52:21.917126 15480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:52:21.917357 15480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:52:21.917397 15480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:52:21.918694 15480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:52:21.918762 15480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:52:21.918812 15480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:52:21.920425 15480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.247.148:41779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:52:21.920686 15480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:52:21.920948 15480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:52:21.926194 15488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:21.926254 15490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:21.926522 15487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:21.926753 15480 server_base.cc:1034] running on GCE node
I20260806 06:52:21.926961 15480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:52:21.927232 15480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:52:21.928390 15480 hybrid_clock.cc:648] HybridClock initialized: now 1785999141928375 us; error 34 us; skew 500 ppm
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:52:21.931324 15480 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:52:21.931875 15480 webserver.cc:460] Webserver started at http://127.3.247.130:41841/ using document root <none> and password file <none>
I20260806 06:52:21.932144 15480 fs_manager.cc:359] Metadata directory not provided
I20260806 06:52:21.932215 15480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:52:21.932353 15480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
Aug 06 06:52:21 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:52:21.958276 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "e62871dd97ed4b01afeb1eb85d2c6514"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "f914972aa3edc90a1e89328a08d54428"
server_key_iv: "e943ba7eab80dc9af1c8a6e058351aa4"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.958739 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "e62871dd97ed4b01afeb1eb85d2c6514"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "f914972aa3edc90a1e89328a08d54428"
server_key_iv: "e943ba7eab80dc9af1c8a6e058351aa4"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.963550 15480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 06:52:21.988224 15497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:21.988482 15480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:52:21.988606 15480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "e62871dd97ed4b01afeb1eb85d2c6514"
format_stamp: "Formatted at 2026-08-06 06:52:21 on dist-test-slave-0kls"
server_key: "f914972aa3edc90a1e89328a08d54428"
server_key_iv: "e943ba7eab80dc9af1c8a6e058351aa4"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:21.988782 15480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:52:22.018769 15480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:52:22.021564 15480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:52:22.021812 15480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:52:22.022096 15480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:52:22.022545 15480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:52:22.022637 15480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:22.022708 15480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:52:22.022766 15480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:22.029765 15480 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:35673
I20260806 06:52:22.030418 15480 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:52:22.031428 15611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:35673 every 8 connection(s)
Aug 06 06:52:22 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999141, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:52:22.040160 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 15480
I20260806 06:52:22.040271 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:52:22.040496 4062 external_mini_cluster.cc:1428] Setting key d33ebd0089c7e32034a318a022ff6e02
I20260806 06:52:22.044399 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.032467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:42481 (local address 127.3.247.190:44613)
0806 06:52:22.032518 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.032521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.033366 (+ 845us) server_negotiation.cc:407] Connection header received
0806 06:52:22.034142 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.034144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.034162 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.034193 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:52:22.035207 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.036013 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.037557 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.037901 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.037949 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.040824 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:52:22.040825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:52:22.040826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:52:22.040834 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:52:22.042774 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.043290 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.043292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.043292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.043351 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.043644 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.043645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.043646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.043780 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:52:22.043822 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.044203 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.044295 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:52:22.045212 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.032178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44613 (local address 127.3.247.130:42481)
0806 06:52:22.033189 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.033251 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.034006 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.034470 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.034475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.034756 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:52:22.035052 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.035060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.036137 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.036138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.037432 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.037436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.038043 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.038044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.038184 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.038803 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:52:22.038814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:52:22.040711 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:52:22.042882 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:22.042890 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:22.042893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:22.043197 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:22.043422 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:22.043423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:22.043423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:22.043566 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:22.043869 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:52:22.043875 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:52:22.044046 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.044792 (+ 746us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.044965 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":80,"threads_started":1}
I20260806 06:52:22.045994 15612 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44613
I20260806 06:52:22.046095 15612 heartbeater.cc:464] Registering TS with master...
I20260806 06:52:22.046283 15612 heartbeater.cc:511] Master 127.3.247.190:44613 requested a full tablet report, sending...
I20260806 06:52:22.046717 15262 ts_manager.cc:194] Registered new tserver with Master: e62871dd97ed4b01afeb1eb85d2c6514 (127.3.247.130:35673)
I20260806 06:52:22.047240 15262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:42481
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:52:22.073346 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:41779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:37437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:52:22.198105 15617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:52:22.198427 15617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:52:22.198480 15617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:52:22.200407 15617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:52:22.200541 15617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:52:22.200635 15617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:52:22.203126 15617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:37437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.247.148:41779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:52:22.203606 15617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:52:22.203934 15617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:52:22.211187 15623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:52:22.211969 15622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:22.212284 15617 server_base.cc:1034] running on GCE node
W20260806 06:52:22.212581 15625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:52:22.212849 15617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:52:22.213155 15617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:52:22.214336 15617 hybrid_clock.cc:648] HybridClock initialized: now 1785999142214310 us; error 36 us; skew 500 ppm
Aug 06 06:52:22 dist-test-slave-0kls krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999142, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:52:22.217775 15617 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:52:22.218362 15617 webserver.cc:460] Webserver started at http://127.3.247.131:37255/ using document root <none> and password file <none>
I20260806 06:52:22.218627 15617 fs_manager.cc:359] Metadata directory not provided
I20260806 06:52:22.218684 15617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:52:22.218806 15617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:52:22 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999142, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:52:22.232285 15617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f"
format_stamp: "Formatted at 2026-08-06 06:52:22 on dist-test-slave-0kls"
server_key: "cd15c7c9439381f381e5232cea8c1052"
server_key_iv: "06a8fe08c7676027a117aecd9b6aac46"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:22.232874 15617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f"
format_stamp: "Formatted at 2026-08-06 06:52:22 on dist-test-slave-0kls"
server_key: "cd15c7c9439381f381e5232cea8c1052"
server_key_iv: "06a8fe08c7676027a117aecd9b6aac46"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:22.238749 15617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 06:52:22.258455 15632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:22.258739 15617 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 06:52:22.258842 15617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f"
format_stamp: "Formatted at 2026-08-06 06:52:22 on dist-test-slave-0kls"
server_key: "cd15c7c9439381f381e5232cea8c1052"
server_key_iv: "06a8fe08c7676027a117aecd9b6aac46"
server_key_version: "kuduclusterkey@0"
I20260806 06:52:22.258960 15617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:52:22.274516 15617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:52:22.276938 15617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:52:22.277127 15617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:52:22.277372 15617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:52:22.277771 15617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:52:22.277902 15617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:22.277976 15617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:52:22.278033 15617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:52:22.285985 15617 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:41601
I20260806 06:52:22.286062 15745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:41601 every 8 connection(s)
I20260806 06:52:22.286448 15617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:52:22.291436 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 15617
I20260806 06:52:22.291538 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:52:22.291712 4062 external_mini_cluster.cc:1428] Setting key e73fede369b9abd9abcf0906c0a63a78
Aug 06 06:52:22 dist-test-slave-0kls krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999142, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:52:22.298444 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.287401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:33569 (local address 127.3.247.190:44613)
0806 06:52:22.287462 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.287464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.287684 (+ 220us) server_negotiation.cc:407] Connection header received
0806 06:52:22.288304 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.288305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.288318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.288391 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:52:22.289110 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.289880 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.291328 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.291653 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.291698 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.294644 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:52:22.294646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:52:22.294646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:52:22.294654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:52:22.296346 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.296825 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.296827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.296827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.296908 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.297218 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.297219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.297219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.297759 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:52:22.297865 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.298228 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.298323 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:52:22.298591 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.287390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44613 (local address 127.3.247.131:33569)
0806 06:52:22.287553 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.287602 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.288189 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.288455 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.288460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.288688 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:52:22.288951 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.288956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.289985 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.289986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.291231 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.291234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.291786 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.291787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.291927 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.292904 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:52:22.292914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:52:22.294541 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:52:22.296440 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:22.296446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:22.296448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:22.296737 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:22.296988 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:52:22.296989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:52:22.296989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:52:22.297127 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:52:22.297882 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:52:22.297888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:52:22.298056 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.298187 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.298344 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 06:52:22.299317 15746 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44613
I20260806 06:52:22.299422 15746 heartbeater.cc:464] Registering TS with master...
I20260806 06:52:22.299599 15746 heartbeater.cc:511] Master 127.3.247.190:44613 requested a full tablet report, sending...
I20260806 06:52:22.299995 15262 ts_manager.cc:194] Registered new tserver with Master: d0cbb40c1d034ed8a9cd3f04ff9f805f (127.3.247.131:41601)
I20260806 06:52:22.300477 15262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:33569
I20260806 06:52:22.303653 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:52:22.317611 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.306237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.3.247.190:44613)
0806 06:52:22.309903 (+ 3666us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.309906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.310174 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:52:22.310268 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.310269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.310279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.310306 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:52:22.311067 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.311792 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.312804 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.313058 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.313116 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.314715 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:52:22.314717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:52:22.314718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:52:22.314725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:52:22.316436 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.316801 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.316802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.316802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.316855 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:52:22.317038 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:52:22.317039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:52:22.317040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:52:22.317151 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:52:22.317216 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.317436 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.317509 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3637}
I20260806 06:52:22.320022 15262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:52:22.322346 15262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:52:22.331947 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.326113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:45855 (local address 127.0.0.1:46854)
0806 06:52:22.327526 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.327533 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.327567 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.327800 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.327801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.327803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:52:22.327862 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.327863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.329272 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.329273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.330381 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.330384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.331653 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.331654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.331740 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.331753 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.331817 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.331867 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"thread_start_us":56,"threads_started":1}
I20260806 06:52:22.332468 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.326687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.3.247.129:45855)
0806 06:52:22.327117 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.327122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.327616 (+ 494us) server_negotiation.cc:407] Connection header received
0806 06:52:22.327654 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.327657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.327673 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.327735 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:52:22.327929 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.329191 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.330476 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.331468 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.331531 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.331588 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.332276 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.332352 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":47,"threads_started":1}
I20260806 06:52:22.332921 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.325739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:35673 (local address 127.0.0.1:42356)
0806 06:52:22.327192 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.327234 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.327309 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.327608 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.327612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.327615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:52:22.327682 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.327683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.329017 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.329018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.330375 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.330378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.332666 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.332667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.332734 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.332738 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.332790 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.332832 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"thread_start_us":84,"threads_started":1}
I20260806 06:52:22.333384 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.326512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.3.247.130:35673)
0806 06:52:22.327175 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.327179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.327335 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:52:22.327400 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.327402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.327420 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.327515 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:52:22.327761 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.328917 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.331675 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.332470 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.332553 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.332624 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.333267 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.333309 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":63,"threads_started":1}
I20260806 06:52:22.333505 15408 tablet_service.cc:1467] Processing CreateTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 (DEFAULT_TABLE table=test-table [id=c1d43aa8920741758e770132446e2167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:52:22.333915 15408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929d46b0f5d4337b4c3211325d3b2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:52:22.334231 15545 tablet_service.cc:1467] Processing CreateTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 (DEFAULT_TABLE table=test-table [id=c1d43aa8920741758e770132446e2167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:52:22.334611 15545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929d46b0f5d4337b4c3211325d3b2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:52:22.337903 15765 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: Bootstrap starting.
I20260806 06:52:22.338622 15765 tablet_bootstrap.cc:654] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: Neither blocks nor log segments found. Creating new log.
I20260806 06:52:22.338971 15765 log.cc:824] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: Log is configured to *not* fsync() on all Append() calls
I20260806 06:52:22.339325 15766 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Bootstrap starting.
I20260806 06:52:22.340238 15766 tablet_bootstrap.cc:654] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Neither blocks nor log segments found. Creating new log.
I20260806 06:52:22.340600 15766 log.cc:824] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Log is configured to *not* fsync() on all Append() calls
I20260806 06:52:22.341338 15765 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: No bootstrap required, opened a new log
I20260806 06:52:22.341724 15765 ts_tablet_manager.cc:1397] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 06:52:22.342294 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.325955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:41601 (local address 127.0.0.1:39928)
0806 06:52:22.327423 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.327431 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.327464 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.338472 (+ 11008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.338473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.338476 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:52:22.338520 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.338522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.339814 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.339815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.340912 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.340915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.342054 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.342055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.342120 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.342124 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.342174 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.342216 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1454,"thread_start_us":75,"threads_started":1}
I20260806 06:52:22.342757 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.326422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.3.247.131:41601)
0806 06:52:22.337919 (+ 11497us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.337924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.337935 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:52:22.337978 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.337980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.337997 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.338120 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:52:22.338627 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.339704 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.341030 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.341875 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.341946 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.342019 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.342637 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.342676 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11452,"thread_start_us":88,"threads_started":1}
I20260806 06:52:22.343556 15662 tablet_service.cc:1467] Processing CreateTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 (DEFAULT_TABLE table=test-table [id=c1d43aa8920741758e770132446e2167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:52:22.343880 15662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9929d46b0f5d4337b4c3211325d3b2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:52:22.344410 15765 raft_consensus.cc:348] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.344563 15765 raft_consensus.cc:374] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:52:22.344590 15765 raft_consensus.cc:729] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4f48a80ed346ab9653c8b1ba564f43, State: Initialized, Role: FOLLOWER
I20260806 06:52:22.344709 15765 consensus_queue.cc:260] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.344930 15765 ts_tablet_manager.cc:1428] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:52:22.345029 15474 heartbeater.cc:503] Master 127.3.247.190:44613 was elected leader, sending a full tablet report...
I20260806 06:52:22.347070 15766 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: No bootstrap required, opened a new log
I20260806 06:52:22.347150 15766 ts_tablet_manager.cc:1397] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 06:52:22.349000 15766 raft_consensus.cc:348] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.349159 15766 raft_consensus.cc:374] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:52:22.349188 15766 raft_consensus.cc:729] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e62871dd97ed4b01afeb1eb85d2c6514, State: Initialized, Role: FOLLOWER
I20260806 06:52:22.349298 15766 consensus_queue.cc:260] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.349532 15766 ts_tablet_manager.cc:1428] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:52:22.349586 15612 heartbeater.cc:503] Master 127.3.247.190:44613 was elected leader, sending a full tablet report...
I20260806 06:52:22.349829 15770 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Bootstrap starting.
I20260806 06:52:22.350858 15770 tablet_bootstrap.cc:654] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Neither blocks nor log segments found. Creating new log.
I20260806 06:52:22.351141 15770 log.cc:824] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:52:22.351912 15770 tablet_bootstrap.cc:492] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: No bootstrap required, opened a new log
I20260806 06:52:22.351979 15770 ts_tablet_manager.cc:1397] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:52:22.355618 15770 raft_consensus.cc:348] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.355787 15770 raft_consensus.cc:374] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:52:22.355823 15770 raft_consensus.cc:729] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cbb40c1d034ed8a9cd3f04ff9f805f, State: Initialized, Role: FOLLOWER
I20260806 06:52:22.355939 15770 consensus_queue.cc:260] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.356196 15770 ts_tablet_manager.cc:1428] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 06:52:22.356417 15746 heartbeater.cc:503] Master 127.3.247.190:44613 was elected leader, sending a full tablet report...
I20260806 06:52:22.417889 15771 raft_consensus.cc:482] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:52:22.418031 15771 raft_consensus.cc:504] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.418403 15771 leader_election.cc:288] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0cbb40c1d034ed8a9cd3f04ff9f805f (127.3.247.131:41601), 6f4f48a80ed346ab9653c8b1ba564f43 (127.3.247.129:45855)
I20260806 06:52:22.422832 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.418695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:41601 (local address 127.3.247.130:55411)
0806 06:52:22.419124 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.419136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.419185 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.419578 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.419579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.419582 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:52:22.419652 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.419654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.420600 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.420601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.421808 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.421811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.422558 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.422559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.422647 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.422651 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.422708 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.422750 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408}
I20260806 06:52:22.423328 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.419310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:55411 (local address 127.3.247.131:41601)
0806 06:52:22.419382 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.419385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.419398 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:52:22.419429 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.419430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.419443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.419521 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:52:22.419726 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.420480 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.421908 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.422442 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.422516 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.422525 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.423221 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.423263 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:52:22.423610 15700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929d46b0f5d4337b4c3211325d3b2a5" candidate_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" is_pre_election: true
I20260806 06:52:22.423763 15700 raft_consensus.cc:2454] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e62871dd97ed4b01afeb1eb85d2c6514 in term 0.
I20260806 06:52:22.424078 15500 leader_election.cc:302] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0cbb40c1d034ed8a9cd3f04ff9f805f, e62871dd97ed4b01afeb1eb85d2c6514; no voters:
I20260806 06:52:22.424216 15771 raft_consensus.cc:2781] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:52:22.424259 15771 raft_consensus.cc:482] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:52:22.424283 15771 raft_consensus.cc:3037] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:52:22.425653 15771 raft_consensus.cc:504] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.426245 15700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929d46b0f5d4337b4c3211325d3b2a5" candidate_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f"
I20260806 06:52:22.426324 15700 raft_consensus.cc:3037] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:52:22.427178 15700 raft_consensus.cc:2454] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e62871dd97ed4b01afeb1eb85d2c6514 in term 1.
I20260806 06:52:22.427390 15500 leader_election.cc:302] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0cbb40c1d034ed8a9cd3f04ff9f805f, e62871dd97ed4b01afeb1eb85d2c6514; no voters:
I20260806 06:52:22.427510 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.419460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:45903 (local address 127.3.247.129:45855)
0806 06:52:22.419564 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.419566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.420464 (+ 898us) server_negotiation.cc:407] Connection header received
0806 06:52:22.423974 (+ 3510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.423976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.423986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.424014 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:52:22.424231 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.424919 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.426220 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.426787 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.426836 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.426843 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.427398 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.427444 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 06:52:22.427750 15774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.418785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:45855 (local address 127.3.247.130:45903)
0806 06:52:22.420392 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.420405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:52:22.423872 (+ 3467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:52:22.424075 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:52:22.424075 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:52:22.424077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:52:22.424161 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.424163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.425018 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.425018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.426119 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:52:22.426122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.426919 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.426920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:52:22.426983 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.426986 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:52:22.427620 (+ 634us) client_negotiation.cc:240] Negotiation successful
0806 06:52:22.427660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1583,"thread_start_us":95,"threads_started":1}
I20260806 06:52:22.427800 15771 leader_election.cc:288] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [CANDIDATE]: Term 1 election: Requested vote from peers d0cbb40c1d034ed8a9cd3f04ff9f805f (127.3.247.131:41601), 6f4f48a80ed346ab9653c8b1ba564f43 (127.3.247.129:45855)
I20260806 06:52:22.427982 15771 raft_consensus.cc:2781] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:52:22.428025 15771 raft_consensus.cc:686] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 1 LEADER]: Becoming Leader. State: Replica: e62871dd97ed4b01afeb1eb85d2c6514, State: Running, Role: LEADER
I20260806 06:52:22.428102 15771 consensus_queue.cc:237] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } }
I20260806 06:52:22.428944 15262 catalog_manager.cc:5495] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 reported cstate change: term changed from 0 to 1, leader changed from <none> to e62871dd97ed4b01afeb1eb85d2c6514 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 } health_report { overall_health: UNKNOWN } } }
I20260806 06:52:22.430358 15428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929d46b0f5d4337b4c3211325d3b2a5" candidate_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" is_pre_election: true
I20260806 06:52:22.430512 15428 raft_consensus.cc:2454] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e62871dd97ed4b01afeb1eb85d2c6514 in term 0.
I20260806 06:52:22.430619 15427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9929d46b0f5d4337b4c3211325d3b2a5" candidate_uuid: "e62871dd97ed4b01afeb1eb85d2c6514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4f48a80ed346ab9653c8b1ba564f43"
I20260806 06:52:22.430660 15427 raft_consensus.cc:3037] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:52:22.431454 15427 raft_consensus.cc:2454] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e62871dd97ed4b01afeb1eb85d2c6514 in term 1.
I20260806 06:52:22.448482 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.445317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.3.247.130:35673)
0806 06:52:22.445354 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:52:22.445356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:52:22.445362 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:52:22.445400 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:52:22.445400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:52:22.445419 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:52:22.445478 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:52:22.445630 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.446412 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.447459 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:52:22.447742 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:52:22.447791 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:52:22.447992 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:52:22.448120 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:52:22.448189 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:52:22.448390 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 06:52:22.448410 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:52:22.451556 15700 raft_consensus.cc:1261] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 1 FOLLOWER]: Refusing update from remote peer e62871dd97ed4b01afeb1eb85d2c6514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:52:22.451545 15427 raft_consensus.cc:1261] T 9929d46b0f5d4337b4c3211325d3b2a5 P 6f4f48a80ed346ab9653c8b1ba564f43 [term 1 FOLLOWER]: Refusing update from remote peer e62871dd97ed4b01afeb1eb85d2c6514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:52:22.451921 15771 consensus_queue.cc:1035] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4f48a80ed346ab9653c8b1ba564f43" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 45855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:52:22.452092 15771 consensus_queue.cc:1035] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0cbb40c1d034ed8a9cd3f04ff9f805f" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 41601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:52:22.456033 15778 mvcc.cc:205] Tried to move back new op lower bound from 7315452487478239232 to 7315452487385993216. Current Snapshot: MvccSnapshot[applied={T|T < 7315452487478239232}]
I20260806 06:52:22.457316 15781 mvcc.cc:205] Tried to move back new op lower bound from 7315452487478239232 to 7315452487385993216. Current Snapshot: MvccSnapshot[applied={T|T < 7315452487478239232}]
I20260806 06:52:22.459678 15779 mvcc.cc:205] Tried to move back new op lower bound from 7315452487478239232 to 7315452487385993216. Current Snapshot: MvccSnapshot[applied={T|T < 7315452487478239232}]
I20260806 06:52:22.460520 15262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:52:22.460608 15262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:52:22.461319 15262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26f0725bc3e24438a35a596600ba201b: Sending DeleteTablet for 3 replicas of tablet 9929d46b0f5d4337b4c3211325d3b2a5
I20260806 06:52:22.461766 15662 tablet_service.cc:1514] Processing DeleteTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:52:22 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:39928
I20260806 06:52:22.461892 15545 tablet_service.cc:1514] Processing DeleteTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:52:22 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:42356
I20260806 06:52:22.461772 15408 tablet_service.cc:1514] Processing DeleteTablet for tablet 9929d46b0f5d4337b4c3211325d3b2a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:52:22 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:46854
I20260806 06:52:22.462579 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 15343
I20260806 06:52:22.467489 15787 tablet_replica.cc:330] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: stopping tablet replica
I20260806 06:52:22.467653 15787 raft_consensus.cc:2229] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:52:22.467862 15787 raft_consensus.cc:2258] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:52:22.468775 15787 ts_tablet_manager.cc:1905] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:52:22.477169 15788 tablet_replica.cc:330] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: stopping tablet replica
I20260806 06:52:22.477308 15788 raft_consensus.cc:2229] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:52:22.477461 15788 raft_consensus.cc:2258] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:52:22.477988 15788 ts_tablet_manager.cc:1905] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:52:22.479662 15788 ts_tablet_manager.cc:1918] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:52:22.479789 15788 log.cc:1196] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/9929d46b0f5d4337b4c3211325d3b2a5
I20260806 06:52:22.479879 15787 ts_tablet_manager.cc:1918] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:52:22.479948 15787 log.cc:1196] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/9929d46b0f5d4337b4c3211325d3b2a5
I20260806 06:52:22.480129 15788 ts_tablet_manager.cc:1939] T 9929d46b0f5d4337b4c3211325d3b2a5 P e62871dd97ed4b01afeb1eb85d2c6514: Deleting consensus metadata
I20260806 06:52:22.480597 15787 ts_tablet_manager.cc:1939] T 9929d46b0f5d4337b4c3211325d3b2a5 P d0cbb40c1d034ed8a9cd3f04ff9f805f: Deleting consensus metadata
I20260806 06:52:22.480599 15248 catalog_manager.cc:4841] TS e62871dd97ed4b01afeb1eb85d2c6514 (127.3.247.130:35673): tablet 9929d46b0f5d4337b4c3211325d3b2a5 (table test-table [id=c1d43aa8920741758e770132446e2167]) successfully deleted
W20260806 06:52:22.480789 15248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:45855 (error 108)
I20260806 06:52:22.481421 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 15480
I20260806 06:52:22.482167 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:52:22.481134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:45855 (local address 127.0.0.1:46862)
0806 06:52:22.482079 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 06:52:22.482130 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:45855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921}
W20260806 06:52:22.483314 15248 catalog_manager.cc:4584] TS 6f4f48a80ed346ab9653c8b1ba564f43 (127.3.247.129:45855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9929d46b0f5d4337b4c3211325d3b2a5: Network error: Client connection negotiation failed: client connection to 127.3.247.129:45855: connect: Connection refused (error 111)
I20260806 06:52:22.483421 15248 catalog_manager.cc:4841] TS d0cbb40c1d034ed8a9cd3f04ff9f805f (127.3.247.131:41601): tablet 9929d46b0f5d4337b4c3211325d3b2a5 (table test-table [id=c1d43aa8920741758e770132446e2167]) successfully deleted
I20260806 06:52:22.486388 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 15617
I20260806 06:52:22.494534 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 15230
I20260806 06:52:22.503175 4062 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:52:22.848104 4062 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:52:22.848276 4062 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:52:23.192965 4062 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:52:23.193 UTC [14816] LOG: received smart shutdown request
2026-08-06 06:52:23.196 UTC [14816] LOG: background worker "logical replication launcher" (PID 14825) exited with exit code 1
2026-08-06 06:52:23.196 UTC [14820] LOG: shutting down
2026-08-06 06:52:23.362 UTC [14816] LOG: database system is shut down
2026-08-06T06:52:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45463 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:52:23 dist-test-slave-0kls krb5kdc[15799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:52:23 dist-test-slave-0kls krb5kdc[15799](info): set up 2 sockets
Aug 06 06:52:23 dist-test-slave-0kls krb5kdc[15799](info): commencing operation
krb5kdc: starting...
W20260806 06:52:24.525851 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:52:24 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:52:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangeradmin_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangeradmin_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangerlookup_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangerlookup_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
I20260806 06:52:24.694576 4062 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktauUQm/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/postgres -l logfile start
2026-08-06 06:52:27.120 UTC [15829] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:52:27.120 UTC [15829] LOG: listening on IPv4 address "127.3.247.148", port 53937
2026-08-06 06:52:27.124 UTC [15829] LOG: listening on Unix socket "/tmp/.s.PGSQL.53937"
2026-08-06 06:52:27.159 UTC [15832] LOG: database system was shut down at 2026-08-06 06:52:25 UTC
2026-08-06 06:52:27.164 UTC [15829] LOG: database system is ready to accept connections
I20260806 06:52:28.117731 4062 mini_postgres.cc:96] Postgres bound to 53937
2026-08-06 06:52:28.123 UTC [15840] FATAL: database "slave" does not exist
127.3.247.148:53937 - accepting connections
I20260806 06:52:28.124398 4062 mini_ranger.cc:161] Starting Ranger...
I20260806 06:52:28.139362 4062 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:52:28.589051 4062 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:52:28.589144 4062 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin
2026-08-06 06:52:28,868 [I] DB FLAVOR :POSTGRES
2026-08-06 06:52:28,869 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:52:28,869 [I] Checking connection..
2026-08-06 06:52:28,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:52:29,136 [I] Checking connection passed.
2026-08-06 06:52:29,136 [I] --------- Verifying version history table ---------
2026-08-06 06:52:29,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:52:29,370 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:52:29,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:52:29,618 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:52:29,619 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:52:29,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:52:29.883 UTC [15904] WARNING: there is no transaction in progress
2026-08-06 06:52:29,889 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:52:29,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:52:30,120 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:52:30,120 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:52:30,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:52:30,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:52:30,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0kls.c.gcp-upstream.internal','N') ;"
2026-08-06 06:52:30,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:52:31,111 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:52:31,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:52:31,352 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:52:31,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:52:31,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:52:31,835 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:52:31,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:52:32.847 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.863 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.880 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.895 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.911 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.929 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.990 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:32.997 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.011 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.021 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.031 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.044 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.053 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.059 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.066 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33.072 UTC [16012] WARNING: there is no transaction in progress
2026-08-06 06:52:33,697 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:52:33,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:52:33,926 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:52:33,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:52:34,173 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:52:34,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:52:34,434 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:52:34,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:52:34,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:52:34,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0kls.c.gcp-upstream.internal';"
2026-08-06 06:52:35,140 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:52:35,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:52:35,395 [I] DB_PATCHES have already been applied
I20260806 06:52:35.401332 4062 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktauUQm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:52:35.401397 4062 mini_ranger.cc:193] Using host: 127.3.247.148
I20260806 06:52:35.404472 4062 mini_ranger.cc:227] Ranger admin URL: http://127.3.247.148:59193
Aug 06, 2026 6:52:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:52:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:52:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:52:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:52:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:52:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:52:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:52:36.533025 4062 mini_ranger.cc:160] Time spent starting Ranger: real 8.409s user 0.000s sys 0.009s
Aug 06, 2026 6:52:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:52:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:52:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:52:56 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:52:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:52:57.785512 4062 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.247.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.247.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangerkms_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.247.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/rangerkms_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.247.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.247.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.247.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/HTTP_127.3.247.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:52:57 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999177, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:52:57.871068 4062 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:52:57.880484 4062 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:52:58.704238 4062 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:52:58.704321 4062 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms
I20260806 06:52:58.704348 4062 mini_ranger_kms.cc:227] Using postgres at 127.3.247.148:53937
2026-08-06 06:52:58,867 [I] DB FLAVOR :POSTGRES
2026-08-06 06:52:58,867 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:52:58,868 [I] Checking connection
2026-08-06 06:52:58,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:52:59,104 [I] connection success
2026-08-06 06:52:59,104 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:52:59,104 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:52:59,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:52:59,349 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:52:59,349 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:52:59,349 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:52:59,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.247.148:53937/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:52:59,608 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:53:00.086689 4062 mini_ranger_kms.cc:310] Created kms service
I20260806 06:53:00.261444 4062 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:53:00.304385 4062 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:53:00 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999177, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:00.714942 4062 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:53:00.715175 4062 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktauUQm/build/release/bin/postgresql.jar:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktauUQm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:53:00.715198 4062 mini_ranger_kms.cc:240] Using host: 127.3.247.148
I20260806 06:53:00.718542 4062 mini_ranger_kms.cc:276] Ranger KMS PID: 16192
I20260806 06:53:00.718639 4062 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.247.148:47353
Aug 06, 2026 6:53:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:53:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:53:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:53:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:53:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:53:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:53:02.956064 4062 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.085s user 0.005s sys 0.140s
I20260806 06:53:02.956166 4062 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:53:02.956210 4062 mini_ranger_kms.cc:401] 127.3.247.148:47353/kms/v1/keys
Aug 06 06:53:03 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785999183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:53:03 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
Aug 06 06:53:03 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.247.148@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
Aug 06 06:53:06 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:53:06.413003 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39215
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39215
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:06.510797 16243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:06.511029 16243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:06.511066 16243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:06.512225 16243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:06.512296 16243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:06.512320 16243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:06.512348 16243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:06.512380 16243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:06.514020 16243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.247.148:47353/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39215
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:06.514297 16243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:06.514509 16243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:06.520630 16248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:06.520746 16249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:06.520864 16251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:06.520807 16243 server_base.cc:1034] running on GCE node
I20260806 06:53:06.521349 16243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:06.521728 16243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:06.522944 16243 hybrid_clock.cc:648] HybridClock initialized: now 1785999186522919 us; error 45 us; skew 500 ppm
Aug 06 06:53:06 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999186, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:06.525703 16243 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:06.526276 16243 webserver.cc:460] Webserver started at http://127.3.247.190:42389/ using document root <none> and password file <none>
I20260806 06:53:06.526531 16243 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:06.526582 16243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:06.526705 16243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:53:06 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999186, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:06.584602 16243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd"
format_stamp: "Formatted at 2026-08-06 06:53:06 on dist-test-slave-0kls"
server_key: "a768d9f47543aa856563cf207323fa2a"
server_key_iv: "321d2a6118714e055a0d929e6f3a28b1"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:06.585024 16243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd"
format_stamp: "Formatted at 2026-08-06 06:53:06 on dist-test-slave-0kls"
server_key: "a768d9f47543aa856563cf207323fa2a"
server_key_iv: "321d2a6118714e055a0d929e6f3a28b1"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:06.586603 16243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:06.601858 16258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:06.602087 16243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:06.602221 16243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd"
format_stamp: "Formatted at 2026-08-06 06:53:06 on dist-test-slave-0kls"
server_key: "a768d9f47543aa856563cf207323fa2a"
server_key_iv: "321d2a6118714e055a0d929e6f3a28b1"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:06.602325 16243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:06.623493 16243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:06.625772 16243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:06.625964 16243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:06.630996 16243 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39215
I20260806 06:53:06.631088 16310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39215 every 8 connection(s)
I20260806 06:53:06.631451 16243 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:06.632176 16311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:06.634560 16311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Bootstrap starting.
I20260806 06:53:06.635149 16311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:06.635447 16311 log.cc:824] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:06.636236 16311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: No bootstrap required, opened a new log
I20260806 06:53:06.638373 16311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:06.638522 16311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:06.638545 16311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d4c4d3d09244aa4a6c0c90a34eaa9cd, State: Initialized, Role: FOLLOWER
I20260806 06:53:06.638698 16311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:06.638788 16311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:06.638835 16311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:06.638892 16311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:06.639477 16311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:06.639560 16311 leader_election.cc:302] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d4c4d3d09244aa4a6c0c90a34eaa9cd; no voters:
I20260806 06:53:06.639750 16311 leader_election.cc:288] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:53:06.639803 16314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:06.639972 16314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 LEADER]: Becoming Leader. State: Replica: 8d4c4d3d09244aa4a6c0c90a34eaa9cd, State: Running, Role: LEADER
I20260806 06:53:06.640129 16314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:06.640265 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16243
I20260806 06:53:06.640376 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:53:06.640595 4062 external_mini_cluster.cc:1428] Setting key 8d42f3de5f6980af4f49e50a5909d000
I20260806 06:53:06.640568 16315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } } }
I20260806 06:53:06.640640 16315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:06.640008 16311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:06.641480 16318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:06.641830 16316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d4c4d3d09244aa4a6c0c90a34eaa9cd. Latest consensus state: current_term: 1 leader_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } } }
I20260806 06:53:06.641960 16316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:06.642105 16318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:06.642406 16311 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:53:06.643810 16318 catalog_manager.cc:1347] Generated new cluster ID: 203fb290f5344050bcd747a35e1c5512
I20260806 06:53:06.643859 16318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:53:06 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:06.653663 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:06.641254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.3.247.190:39215)
0806 06:53:06.642228 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:06.642237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:06.642271 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:53:06.642404 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:06.642412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:06.642471 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:06.642616 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:06.643526 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:06.644522 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:06.645592 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:06.646221 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:06.646304 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:06.649415 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:06.649419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:06.649420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:06.649432 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:06.651559 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:06.651998 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:06.652002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:06.652002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:06.652123 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:06.652338 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:06.652339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:06.652339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:06.652651 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:06.652779 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:06.653300 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 06:53:06.653452 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":93,"threads_started":1}
I20260806 06:53:06.653726 16318 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:53:06.654412 16318 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:06.661177 16318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Generated new TSK 0
I20260806 06:53:06.661725 16318 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:53:07.337725 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:07.454316 16356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:07.454614 16356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:07.454653 16356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:07.456418 16356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:07.456475 16356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:07.456516 16356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:07.458719 16356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:07.459123 16356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:07.459440 16356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:07.465768 16362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:07.465824 16361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:07.465912 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:07.467113 16356 server_base.cc:1034] running on GCE node
I20260806 06:53:07.467351 16356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:07.467674 16356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:07.468874 16356 hybrid_clock.cc:648] HybridClock initialized: now 1785999187468860 us; error 47 us; skew 500 ppm
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:07.471992 16356 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:07.472452 16356 webserver.cc:460] Webserver started at http://127.3.247.129:46839/ using document root <none> and password file <none>
I20260806 06:53:07.472689 16356 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:07.472759 16356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:07.472915 16356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:07.488018 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "23132a4c62eb45bfb8b22e8c58fa91f3"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "4085f7fe1b71cc4824e334eb9ad119ef"
server_key_iv: "fcd237b48cf3a1e2b825856de198972e"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.488634 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "23132a4c62eb45bfb8b22e8c58fa91f3"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "4085f7fe1b71cc4824e334eb9ad119ef"
server_key_iv: "fcd237b48cf3a1e2b825856de198972e"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.493121 16356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 06:53:07.507175 16373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.507417 16356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:07.507530 16356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "23132a4c62eb45bfb8b22e8c58fa91f3"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "4085f7fe1b71cc4824e334eb9ad119ef"
server_key_iv: "fcd237b48cf3a1e2b825856de198972e"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.507627 16356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:07.526516 16356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:07.529217 16356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:07.529456 16356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:07.529816 16356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:07.530308 16356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:07.530405 16356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.530506 16356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:07.530633 16356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.537581 16356 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:43533
I20260806 06:53:07.538233 16356 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:07.545130 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16356
I20260806 06:53:07.545261 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:53:07.545466 4062 external_mini_cluster.cc:1428] Setting key 6aafddd4315be6620ec91ec1b0fb33c5
I20260806 06:53:07.547041 16486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:43533 every 8 connection(s)
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:07.559345 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:07.547678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.129:49305)
0806 06:53:07.548123 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:07.548185 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:07.548805 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:07.549329 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:07.549334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:07.549592 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:07.549896 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:07.549903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.550891 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.550892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:07.552133 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:07.552135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.552705 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.552706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:07.552852 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:07.553556 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:07.553567 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:07.555310 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:07.557518 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:07.557527 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:07.557530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:07.557888 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:07.558128 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:07.558129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:07.558129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:07.558276 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:07.558598 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:07.558606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:07.558786 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 06:53:07.558905 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 06:53:07.559076 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":84,"threads_started":1}
I20260806 06:53:07.560005 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:07.547675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49305 (local address 127.3.247.190:39215)
0806 06:53:07.549062 (+ 1387us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:07.549064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:07.549072 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:07.549096 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:07.549097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:07.549109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:07.549191 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:07.550009 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.550788 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.552240 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.552551 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.552618 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:07.555429 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:07.555430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:07.555431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:07.555438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:07.557420 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:07.557979 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:07.557980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:07.557980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:07.558051 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:07.558351 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:07.558352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:07.558353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:07.558499 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:07.558566 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:07.559800 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0806 06:53:07.559909 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":80,"threads_started":1}
I20260806 06:53:07.560806 16487 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:07.560923 16487 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:07.561131 16487 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:53:07.561771 16275 ts_manager.cc:194] Registered new tserver with Master: 23132a4c62eb45bfb8b22e8c58fa91f3 (127.3.247.129:43533)
I20260806 06:53:07.562706 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:49305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:53:07.608925 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 06:53:07.733546 16494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:07.734000 16494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:07.734084 16494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:07.735960 16494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:07.736186 16494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:07.736286 16494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:07.738790 16494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:07.739282 16494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:07.739658 16494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:07.746628 16503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:07.746690 16501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:07.746999 16500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:07.748149 16494 server_base.cc:1034] running on GCE node
I20260806 06:53:07.748373 16494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:07.748660 16494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:07.749859 16494 hybrid_clock.cc:648] HybridClock initialized: now 1785999187749836 us; error 39 us; skew 500 ppm
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:07.753444 16494 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:07.754206 16494 webserver.cc:460] Webserver started at http://127.3.247.130:35577/ using document root <none> and password file <none>
I20260806 06:53:07.754537 16494 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:07.754679 16494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:07.754890 16494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:07.771399 16494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "329e8ef85d354fcc87e49350440fd29a"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "3d8fe792f92e2cd3a935392b917c2a86"
server_key_iv: "f91e4f6468d79f5484b157cf2d0ca02b"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.771881 16494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "329e8ef85d354fcc87e49350440fd29a"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "3d8fe792f92e2cd3a935392b917c2a86"
server_key_iv: "f91e4f6468d79f5484b157cf2d0ca02b"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.773609 16494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:07.787883 16510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.788096 16494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:07.788203 16494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "329e8ef85d354fcc87e49350440fd29a"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "3d8fe792f92e2cd3a935392b917c2a86"
server_key_iv: "f91e4f6468d79f5484b157cf2d0ca02b"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:07.788317 16494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:07.813509 16494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:07.815937 16494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:07.816079 16494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:07.816298 16494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:07.816690 16494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:07.816742 16494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.816776 16494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:07.816836 16494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:07.823428 16494 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:37005
I20260806 06:53:07.823941 16494 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:07.830852 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16494
I20260806 06:53:07.830960 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:53:07.831151 4062 external_mini_cluster.cc:1428] Setting key 17a5cdb8d30406f9831f1301bb5600ac
I20260806 06:53:07.831517 16623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:37005 every 8 connection(s)
Aug 06 06:53:07 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999187, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:07.842417 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:07.831154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.130:43737)
0806 06:53:07.831859 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:07.831900 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:07.832556 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:07.832950 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:07.832955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:07.833209 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:07.833532 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:07.833540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.834567 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.834568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:07.835612 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:07.835614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.836193 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.836194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:07.836354 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:07.836996 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:07.837006 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:07.838797 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:07.840696 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:07.840703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:07.840706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:07.841004 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:07.841231 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:07.841232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:07.841232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:07.841365 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:07.841662 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:07.841668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:07.841878 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 06:53:07.841991 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:53:07.842152 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":101,"threads_started":1}
I20260806 06:53:07.843137 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:07.831539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:43737 (local address 127.3.247.190:39215)
0806 06:53:07.831632 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:07.831634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:07.831986 (+ 352us) server_negotiation.cc:407] Connection header received
0806 06:53:07.832690 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:07.832692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:07.832721 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:07.832754 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:07.833648 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.834448 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.835735 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:07.836036 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:07.836107 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:07.838910 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:07.838911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:07.838912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:07.838920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:07.840598 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:07.841089 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:07.841091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:07.841091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:07.841153 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:07.841437 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:07.841438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:07.841438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:07.841578 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:07.841638 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:07.842942 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0806 06:53:07.843037 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 06:53:07.843680 16624 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:07.843780 16624 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:07.843967 16624 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:07.844311 16275 ts_manager.cc:194] Registered new tserver with Master: 329e8ef85d354fcc87e49350440fd29a (127.3.247.130:37005)
I20260806 06:53:07.844787 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:43737
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:53:07.881525 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:08.004384 16630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:08.004899 16630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:08.005053 16630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:08.010177 16630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:08.010324 16630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:08.010416 16630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:08.013695 16630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:08.014329 16630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:08.014685 16630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:08.021489 16636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.021471 16635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.021924 16638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.022982 16630 server_base.cc:1034] running on GCE node
I20260806 06:53:08.023312 16630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:08.023581 16630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:08.031687 16630 hybrid_clock.cc:648] HybridClock initialized: now 1785999188031659 us; error 44 us; skew 500 ppm
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:08.034505 16630 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:08.034977 16630 webserver.cc:460] Webserver started at http://127.3.247.131:36723/ using document root <none> and password file <none>
I20260806 06:53:08.035225 16630 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:08.035296 16630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:08.035439 16630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:08.048782 16630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "b2a3c7f356be41dda5251b13491435bb"
format_stamp: "Formatted at 2026-08-06 06:53:08 on dist-test-slave-0kls"
server_key: "7d5018e426672a070e037472a77804b5"
server_key_iv: "e090382287171574eb0fe22a02cdd405"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.049391 16630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "b2a3c7f356be41dda5251b13491435bb"
format_stamp: "Formatted at 2026-08-06 06:53:08 on dist-test-slave-0kls"
server_key: "7d5018e426672a070e037472a77804b5"
server_key_iv: "e090382287171574eb0fe22a02cdd405"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.052064 16630 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:08.089954 16645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.090173 16630 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.001s sys 0.000s
I20260806 06:53:08.090273 16630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b2a3c7f356be41dda5251b13491435bb"
format_stamp: "Formatted at 2026-08-06 06:53:08 on dist-test-slave-0kls"
server_key: "7d5018e426672a070e037472a77804b5"
server_key_iv: "e090382287171574eb0fe22a02cdd405"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.090384 16630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:08.142554 16630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:08.145012 16630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:08.145162 16630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:08.145422 16630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:08.145838 16630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:08.145890 16630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.145927 16630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:08.145951 16630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.154155 16630 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:43325
I20260806 06:53:08.154589 16630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:53:08.154945 16758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:43325 every 8 connection(s)
I20260806 06:53:08.160773 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16630
I20260806 06:53:08.160884 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:53:08.161077 4062 external_mini_cluster.cc:1428] Setting key 577a32ce0c4d002d24295e588d522e9f
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:08.169298 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.155629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:45073 (local address 127.3.247.190:39215)
0806 06:53:08.155681 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:08.155683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:08.158313 (+ 2630us) server_negotiation.cc:407] Connection header received
0806 06:53:08.159574 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:08.159576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:08.159590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:08.159621 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:08.160508 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.161272 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.162894 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.163206 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.163293 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.165949 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:08.165951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:08.165951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:08.165959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:08.167725 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.168217 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.168218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.168219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.168281 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.168540 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.168540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.168541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.168671 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:08.168730 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:08.169106 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 06:53:08.169202 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:53:08.169734 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.155635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.131:45073)
0806 06:53:08.158147 (+ 2512us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:08.158202 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:08.159422 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:08.159832 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:08.159837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:08.160104 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:08.160391 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.160398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.161392 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.161393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.162786 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.162789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.163329 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.163330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.163474 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.164123 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:08.164134 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:08.165852 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:08.167813 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.167820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.167822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.168123 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.168347 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.168348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.168348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.168473 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.168761 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:08.168768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:08.168929 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 06:53:08.169030 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:53:08.169180 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2469,"thread_start_us":103,"threads_started":1}
I20260806 06:53:08.170461 16759 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:08.170567 16759 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.170760 16759 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:08.171145 16275 ts_manager.cc:194] Registered new tserver with Master: b2a3c7f356be41dda5251b13491435bb (127.3.247.131:43325)
I20260806 06:53:08.171690 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:45073
I20260806 06:53:08.172655 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:08.173349 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16356
I20260806 06:53:08.179384 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16494
I20260806 06:53:08.185155 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16630
I20260806 06:53:08.191027 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16243
I20260806 06:53:08.197937 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39215
--webserver_interface=127.3.247.190
--webserver_port=42389
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39215
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:08.283351 16762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:08.283560 16762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:08.283583 16762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:08.284785 16762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:08.284834 16762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:08.284852 16762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:08.284865 16762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:08.284920 16762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:08.286550 16762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39215
--ranger_config_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=42389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:08.286823 16762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:08.287055 16762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:08.292059 16767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.292088 16768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.292217 16770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.292336 16762 server_base.cc:1034] running on GCE node
I20260806 06:53:08.292608 16762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:08.292922 16762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:08.294096 16762 hybrid_clock.cc:648] HybridClock initialized: now 1785999188294087 us; error 36 us; skew 500 ppm
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:08.296554 16762 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:08.297024 16762 webserver.cc:460] Webserver started at http://127.3.247.190:42389/ using document root <none> and password file <none>
I20260806 06:53:08.297210 16762 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:08.297258 16762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:08.376487 16762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:08.377303 16778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.377521 16762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:08.377624 16762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd"
format_stamp: "Formatted at 2026-08-06 06:53:06 on dist-test-slave-0kls"
server_key: "a768d9f47543aa856563cf207323fa2a"
server_key_iv: "321d2a6118714e055a0d929e6f3a28b1"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.378083 16762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:08.394259 16762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:08.396442 16762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:08.396613 16762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:08.401073 16762 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39215
I20260806 06:53:08.401117 16830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39215 every 8 connection(s)
I20260806 06:53:08.401489 16762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:08.403060 16831 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:53:08.403906 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Bootstrap starting.
I20260806 06:53:08.404146 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16762
I20260806 06:53:08.404438 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:43533
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:39215
--webserver_port=46839
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:08.414201 16831 log.cc:824] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:08.416616 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:08.416960 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Bootstrap complete.
I20260806 06:53:08.419552 16831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:08.419791 16831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d4c4d3d09244aa4a6c0c90a34eaa9cd, State: Initialized, Role: FOLLOWER
I20260806 06:53:08.419986 16831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:08.420059 16831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:08.420110 16831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:08.420158 16831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:53:08.421207 16831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:08.421325 16831 leader_election.cc:302] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d4c4d3d09244aa4a6c0c90a34eaa9cd; no voters:
I20260806 06:53:08.421523 16831 leader_election.cc:288] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:53:08.421558 16835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:53:08.421671 16835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [term 2 LEADER]: Becoming Leader. State: Replica: 8d4c4d3d09244aa4a6c0c90a34eaa9cd, State: Running, Role: LEADER
I20260806 06:53:08.421805 16831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:08.421768 16835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } }
I20260806 06:53:08.422055 16835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } } }
I20260806 06:53:08.422125 16835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:08.422245 16835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d4c4d3d09244aa4a6c0c90a34eaa9cd. Latest consensus state: current_term: 2 leader_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4c4d3d09244aa4a6c0c90a34eaa9cd" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39215 } } }
I20260806 06:53:08.422317 16835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:08.422575 16839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:08.423161 16839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:08.423313 16839 catalog_manager.cc:1259] Loaded cluster ID: 203fb290f5344050bcd747a35e1c5512
I20260806 06:53:08.423357 16839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:53:08.423501 16831 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:53:08.427887 16839 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:08.428560 16839 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Loaded TSK: 0
I20260806 06:53:08.428905 16839 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:53:08.522570 16833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:08.522931 16833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:08.522969 16833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:08.524771 16833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:08.524834 16833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:08.524876 16833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:08.527050 16833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:43533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=46839
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:08.527314 16833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:08.527599 16833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:08.534296 16869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.534282 16868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.534286 16871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.535419 16833 server_base.cc:1034] running on GCE node
I20260806 06:53:08.535760 16833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:08.536057 16833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:08.537242 16833 hybrid_clock.cc:648] HybridClock initialized: now 1785999188537215 us; error 40 us; skew 500 ppm
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:08.540659 16833 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:08.541244 16833 webserver.cc:460] Webserver started at http://127.3.247.129:46839/ using document root <none> and password file <none>
I20260806 06:53:08.541522 16833 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:08.541610 16833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:08.573644 16833 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.001s
I20260806 06:53:08.574391 16879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.574538 16833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.574641 16833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "23132a4c62eb45bfb8b22e8c58fa91f3"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "4085f7fe1b71cc4824e334eb9ad119ef"
server_key_iv: "fcd237b48cf3a1e2b825856de198972e"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.575779 16833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:08.596413 16833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:08.598829 16833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:08.598976 16833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:08.599206 16833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:08.599527 16833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:08.599568 16833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.599597 16833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:08.599618 16833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.605620 16833 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:43533
I20260806 06:53:08.606976 16833 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:08.607991 16992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:43533 every 8 connection(s)
I20260806 06:53:08.613912 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16833
I20260806 06:53:08.614290 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:37005
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:39215
--webserver_port=35577
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:08.623016 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.609338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.129:33695)
0806 06:53:08.609935 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:08.609981 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:08.610740 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:08.611395 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:08.611400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:08.611697 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:08.612003 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.612011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.613909 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.613910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.615254 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.615258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.616294 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.616296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.616438 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.617093 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:08.617104 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:08.618858 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:08.621095 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.621101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.621104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.621395 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.621658 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.621658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.621659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.621775 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.622324 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:08.622330 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:08.622478 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:53:08.622572 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:53:08.622730 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":81,"threads_started":1}
I20260806 06:53:08.623884 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.609055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:33695 (local address 127.3.247.190:39215)
0806 06:53:08.609842 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:08.609852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:08.610528 (+ 676us) server_negotiation.cc:407] Connection header received
0806 06:53:08.611018 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:08.611025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:08.611083 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:08.611216 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:08.612183 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.613725 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.615359 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.616071 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.616177 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.619019 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:08.619028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:08.619028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:08.619050 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:08.620988 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.621486 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.621490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.621490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.621593 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.621902 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.621903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.621903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.622227 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:08.622298 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:08.623595 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0806 06:53:08.623724 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":83,"threads_started":1}
I20260806 06:53:08.624548 16993 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:08.624663 16993 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.624854 16993 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:08.625758 16993 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:39215 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:53:08.625864 16993 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.625924 16993 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:08.626305 16993 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.626381 16993 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:08.626670 16993 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:53:08.728091 16997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:08.728382 16997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:08.728421 16997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:08.730216 16997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:08.730274 16997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:08.730316 16997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:08.732542 16997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:37005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=35577
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:08.732812 16997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:08.733089 16997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:08.739684 17003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.739786 17005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.739895 16997 server_base.cc:1034] running on GCE node
W20260806 06:53:08.740126 17002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.740373 16997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:08.740653 16997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:08.741847 16997 hybrid_clock.cc:648] HybridClock initialized: now 1785999188741824 us; error 34 us; skew 500 ppm
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:08.748006 16997 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:08.748530 16997 webserver.cc:460] Webserver started at http://127.3.247.130:35577/ using document root <none> and password file <none>
I20260806 06:53:08.748785 16997 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:08.748899 16997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:08.766119 16997 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:53:08.767138 17013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.767338 16997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:08.767431 16997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "329e8ef85d354fcc87e49350440fd29a"
format_stamp: "Formatted at 2026-08-06 06:53:07 on dist-test-slave-0kls"
server_key: "3d8fe792f92e2cd3a935392b917c2a86"
server_key_iv: "f91e4f6468d79f5484b157cf2d0ca02b"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.767829 16997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:08.788877 16997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:08.791334 16997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:08.791502 16997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:08.791802 16997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:08.792155 16997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:08.792189 16997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.792212 16997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:08.792225 16997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.799750 16997 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:37005
I20260806 06:53:08.799916 17126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:37005 every 8 connection(s)
I20260806 06:53:08.800251 16997 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:08.804028 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 16997
I20260806 06:53:08.804352 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.247.148:47353/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:43325
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:39215
--webserver_port=36723
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:08.811172 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.801473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:59899 (local address 127.3.247.190:39215)
0806 06:53:08.801529 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:08.801531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:08.801545 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:53:08.802206 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:08.802209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:08.802225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:08.802295 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:08.803053 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.803810 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.804971 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.805190 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.805271 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.807950 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:08.807952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:08.807952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:08.807960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:08.809528 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.810067 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.810069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.810069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.810129 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:08.810420 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:08.810420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:08.810421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:08.810548 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:08.810620 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:08.810989 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 06:53:08.811075 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:53:08.811324 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.801139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.130:59899)
0806 06:53:08.801350 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:08.801403 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:08.802069 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:08.802375 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:08.802379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:08.802638 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:08.802931 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.802939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.803937 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.803938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.804849 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:08.804851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:08.805300 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:08.805301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:08.805464 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:08.806135 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:08.806146 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:08.807816 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:08.809628 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.809635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.809638 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.809988 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.810218 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:08.810219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:08.810219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:08.810339 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:08.810621 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:08.810628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:08.810777 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 06:53:08.810881 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:53:08.811060 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260806 06:53:08.811957 17127 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:08.812083 17127 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.812274 17127 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:08.812904 17127 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:39215 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:53:08.812986 17127 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.813057 17127 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:08.813333 17127 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:08.813423 17127 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:08.813686 17127 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:53:08.925125 17130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:08.925413 17130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:08.925451 17130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:08.927258 17130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:08.927318 17130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:08.927361 17130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:08.929519 17130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.247.148:47353/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:43325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=36723
--tserver_master_addrs=127.3.247.190:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:08.929776 17130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:08.930217 17130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:08.937286 17136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.938342 17137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:08.938601 17139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:08.939374 17130 server_base.cc:1034] running on GCE node
I20260806 06:53:08.939608 17130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:08.939898 17130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:08.941077 17130 hybrid_clock.cc:648] HybridClock initialized: now 1785999188941057 us; error 31 us; skew 500 ppm
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:08.944473 17130 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:08.945092 17130 webserver.cc:460] Webserver started at http://127.3.247.131:36723/ using document root <none> and password file <none>
I20260806 06:53:08.945354 17130 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:08.945449 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:53:08 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:08.967224 17130 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:08.968284 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.968506 17130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:08.968617 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b2a3c7f356be41dda5251b13491435bb"
format_stamp: "Formatted at 2026-08-06 06:53:08 on dist-test-slave-0kls"
server_key: "7d5018e426672a070e037472a77804b5"
server_key_iv: "e090382287171574eb0fe22a02cdd405"
server_key_version: "kuduclusterkey@0"
I20260806 06:53:08.969013 17130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:08.986246 17130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:08.988749 17130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:08.988943 17130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:08.989204 17130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:08.989578 17130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:08.989632 17130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.989666 17130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:08.989693 17130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:08.997216 17130 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:43325
I20260806 06:53:08.997706 17130 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:53:08.998440 17260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:43325 every 8 connection(s)
I20260806 06:53:09.003887 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17130
Aug 06 06:53:09 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999188, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:09.011193 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.999240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:48925 (local address 127.3.247.190:39215)
0806 06:53:08.999287 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:08.999290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:09.000331 (+ 1041us) server_negotiation.cc:407] Connection header received
0806 06:53:09.001039 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:09.001040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:09.001055 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:09.001122 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:09.002113 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.002804 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.004773 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.005092 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.005137 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:09.007729 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:09.007730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:09.007731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:09.007738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:09.009383 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:09.009945 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:09.009946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:09.009946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:09.010012 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:09.010335 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:09.010337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:09.010337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:09.010473 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:09.010544 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:09.010972 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 06:53:09.011075 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:53:09.011317 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:08.999111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:39215 (local address 127.3.247.131:48925)
0806 06:53:09.000197 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:09.000239 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:09.000913 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:09.001196 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:09.001200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:09.001446 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:09.001978 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:09.001986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.002924 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.002925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:09.004675 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:09.004678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.005257 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.005258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:09.005396 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:09.005998 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:09.006008 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:09.007630 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:09.009480 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:09.009486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:09.009488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:09.009859 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:09.010092 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:09.010093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:09.010093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:09.010248 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:09.010585 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:09.010591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:09.010774 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 06:53:09.010904 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 06:53:09.011077 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":826,"threads_started":1}
I20260806 06:53:09.011756 17261 heartbeater.cc:347] Connected to a master server at 127.3.247.190:39215
I20260806 06:53:09.011868 17261 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:09.012072 17261 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:09.012784 17261 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:39215 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:53:09.012864 17261 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:09.012923 17261 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:09.013207 17261 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:09.013298 17261 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
W20260806 06:53:09.013618 17261 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:53:09.014823 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:09.004452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.3.247.190:39215)
0806 06:53:09.005897 (+ 1445us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:09.005899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:09.005906 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:53:09.006557 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:09.006558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:09.006571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:09.006634 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:09.007441 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.008200 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.009281 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:09.009586 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:09.009646 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:09.011354 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:09.011355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:09.011355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:09.011362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:09.013699 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:09.014077 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:09.014078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:09.014078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:09.014138 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:09.014308 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:09.014309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:09.014309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:09.014431 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:09.014592 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:09.014635 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:53:09.014718 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1423,"thread_start_us":60,"threads_started":1}
Aug 06 06:53:09 dist-test-slave-0kls krb5kdc[15799](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785999189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:09.626859 16993 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:09.627017 16993 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:09.627413 16795 ts_manager.cc:194] Registered new tserver with Master: 23132a4c62eb45bfb8b22e8c58fa91f3 (127.3.247.129:43533)
I20260806 06:53:09.628497 16795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:33695
I20260806 06:53:09.813881 17127 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:09.814060 17127 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:09.814429 16795 ts_manager.cc:194] Registered new tserver with Master: 329e8ef85d354fcc87e49350440fd29a (127.3.247.130:37005)
I20260806 06:53:09.814986 16795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:59899
Aug 06 06:53:09 dist-test-slave-0kls krb5kdc[15799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999189, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for HTTP/127.3.247.148@KRBTEST.COM
I20260806 06:53:10.013849 17261 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:10.014006 17261 heartbeater.cc:511] Master 127.3.247.190:39215 requested a full tablet report, sending...
I20260806 06:53:10.014379 16795 ts_manager.cc:194] Registered new tserver with Master: b2a3c7f356be41dda5251b13491435bb (127.3.247.131:43325)
I20260806 06:53:10.014918 16795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:48925
I20260806 06:53:10.018021 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:10.027195 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.019021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.3.247.190:39215)
0806 06:53:10.019267 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.019269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.019277 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:10.019298 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.019299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.019310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.019392 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:10.020090 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.020872 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.021988 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.022303 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.022393 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.023905 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:10.023906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:10.023906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:10.023914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:10.025752 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:10.026205 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:10.026206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:10.026206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:10.026276 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:10.026505 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:10.026506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:10.026507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:10.026651 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:10.026756 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.026880 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.026969 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":67,"threads_started":1}
I20260806 06:53:10.029448 16795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:53:10.031986 16795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:53:10.040103 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:43325 (local address 127.0.0.1:57726)
0806 06:53:10.035544 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.035587 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:10.035669 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:10.036046 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:10.036049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:10.036052 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:10.036124 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.036126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.037416 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.037417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.038786 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.038789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.039775 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.039776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.039864 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.039880 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:53:10.039958 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:53:10.040012 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":89,"threads_started":1}
I20260806 06:53:10.040417 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.3.247.131:43325)
0806 06:53:10.035536 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.035541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.035668 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:53:10.035827 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.035830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.035847 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.035928 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:10.036217 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.037311 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.038891 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.039682 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.039766 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.039829 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.040275 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.040323 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":49,"threads_started":1}
I20260806 06:53:10.041074 17195 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2f2fdca756449ba52c0753798b4caa (DEFAULT_TABLE table=test-table [id=5df83ee779e74432a47bf7ea81e140a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:10.041431 17195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2f2fdca756449ba52c0753798b4caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:10.046541 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:37005 (local address 127.0.0.1:47898)
0806 06:53:10.040654 (+ 5511us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.040665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:10.040706 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:10.042213 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:10.042214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:10.042216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:10.042249 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.042250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.043852 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.043853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.044933 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.044937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.045991 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.045992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.046055 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.046059 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:10.046111 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:53:10.046153 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5498,"thread_start_us":84,"threads_started":1}
I20260806 06:53:10.047022 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.3.247.130:37005)
0806 06:53:10.041945 (+ 6696us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.041951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.041965 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:53:10.042001 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.042004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.042021 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.042109 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:10.042351 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.043753 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.045034 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.045822 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.045894 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.045956 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.046897 (+ 941us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.046943 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6655,"thread_start_us":50,"threads_started":1}
I20260806 06:53:10.047361 17295 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Bootstrap starting.
I20260806 06:53:10.047690 17061 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2f2fdca756449ba52c0753798b4caa (DEFAULT_TABLE table=test-table [id=5df83ee779e74432a47bf7ea81e140a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:10.048080 17061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2f2fdca756449ba52c0753798b4caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:10.048128 17295 tablet_bootstrap.cc:654] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:10.048291 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43533 (local address 127.0.0.1:54574)
0806 06:53:10.044340 (+ 8957us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.044354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:10.044443 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:10.044739 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:10.044740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:10.044742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:10.044790 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.044791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.045756 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.045757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.046935 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.046938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.048033 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.048034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.048112 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.048116 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:10.048165 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:53:10.048208 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8919,"thread_start_us":65,"threads_started":1}
I20260806 06:53:10.048580 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.035488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.3.247.129:43533)
0806 06:53:10.041960 (+ 6472us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.041966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.044513 (+ 2547us) server_negotiation.cc:407] Connection header received
0806 06:53:10.044556 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.044558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.044576 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.044650 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:10.044874 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.045656 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.047036 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.047823 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.047935 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.047997 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.048455 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.048503 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6432,"thread_start_us":43,"threads_started":1}
I20260806 06:53:10.048611 17295 log.cc:824] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:10.049188 16927 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2f2fdca756449ba52c0753798b4caa (DEFAULT_TABLE table=test-table [id=5df83ee779e74432a47bf7ea81e140a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:10.049502 16927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2f2fdca756449ba52c0753798b4caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:10.051683 17297 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Bootstrap starting.
I20260806 06:53:10.052313 17297 tablet_bootstrap.cc:654] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:10.052623 17297 log.cc:824] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:10.053226 17298 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: Bootstrap starting.
I20260806 06:53:10.053478 17295 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: No bootstrap required, opened a new log
I20260806 06:53:10.053562 17295 ts_tablet_manager.cc:1397] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 06:53:10.053879 17298 tablet_bootstrap.cc:654] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:10.054194 17298 log.cc:824] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:10.054986 17297 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: No bootstrap required, opened a new log
I20260806 06:53:10.055058 17297 ts_tablet_manager.cc:1397] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:53:10.055137 17298 tablet_bootstrap.cc:492] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: No bootstrap required, opened a new log
I20260806 06:53:10.055208 17298 ts_tablet_manager.cc:1397] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:53:10.057583 17297 raft_consensus.cc:348] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.057762 17297 raft_consensus.cc:374] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:10.057843 17297 raft_consensus.cc:729] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329e8ef85d354fcc87e49350440fd29a, State: Initialized, Role: FOLLOWER
I20260806 06:53:10.057993 17297 consensus_queue.cc:260] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.058267 17297 ts_tablet_manager.cc:1428] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:53:10.058391 17127 heartbeater.cc:503] Master 127.3.247.190:39215 was elected leader, sending a full tablet report...
I20260806 06:53:10.058401 17298 raft_consensus.cc:348] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.058564 17298 raft_consensus.cc:374] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:10.058602 17298 raft_consensus.cc:729] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23132a4c62eb45bfb8b22e8c58fa91f3, State: Initialized, Role: FOLLOWER
I20260806 06:53:10.058746 17298 consensus_queue.cc:260] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.059013 17298 ts_tablet_manager.cc:1428] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 06:53:10.059134 16993 heartbeater.cc:503] Master 127.3.247.190:39215 was elected leader, sending a full tablet report...
I20260806 06:53:10.061025 17295 raft_consensus.cc:348] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.061208 17295 raft_consensus.cc:374] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:10.061244 17295 raft_consensus.cc:729] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a3c7f356be41dda5251b13491435bb, State: Initialized, Role: FOLLOWER
I20260806 06:53:10.061390 17295 consensus_queue.cc:260] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.061712 17261 heartbeater.cc:503] Master 127.3.247.190:39215 was elected leader, sending a full tablet report...
I20260806 06:53:10.061695 17295 ts_tablet_manager.cc:1428] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260806 06:53:10.076062 17302 raft_consensus.cc:482] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:10.076224 17302 raft_consensus.cc:504] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.077507 17302 leader_election.cc:288] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2a3c7f356be41dda5251b13491435bb (127.3.247.131:43325), 329e8ef85d354fcc87e49350440fd29a (127.3.247.130:37005)
I20260806 06:53:10.081233 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.077435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:35181 (local address 127.3.247.131:43325)
0806 06:53:10.077484 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.077486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.077635 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:53:10.077774 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.077775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.077812 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.077849 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:10.078066 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.078769 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.080250 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.080788 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.080834 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.080842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.081108 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.081148 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:53:10.081316 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.077442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:51429 (local address 127.3.247.130:37005)
0806 06:53:10.077484 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.077486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.077715 (+ 229us) server_negotiation.cc:407] Connection header received
0806 06:53:10.077851 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.077853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.077864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.077895 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:10.078110 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.078853 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.080297 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.080858 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.080910 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.080918 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.081204 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.081248 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:53:10.081550 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.077224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:37005 (local address 127.3.247.129:51429)
0806 06:53:10.077651 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.077662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:10.077741 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:10.077964 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:10.077965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:10.077967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:10.078039 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.078041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.078951 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.078952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.080196 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.080199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.081011 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.081011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.081091 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.081094 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:10.081422 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 06:53:10.081463 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":117,"threads_started":1}
I20260806 06:53:10.081890 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2f2fdca756449ba52c0753798b4caa" candidate_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329e8ef85d354fcc87e49350440fd29a" is_pre_election: true
I20260806 06:53:10.082047 17081 raft_consensus.cc:2454] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23132a4c62eb45bfb8b22e8c58fa91f3 in term 0.
I20260806 06:53:10.082261 16882 leader_election.cc:302] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23132a4c62eb45bfb8b22e8c58fa91f3, 329e8ef85d354fcc87e49350440fd29a; no voters:
I20260806 06:53:10.082379 17302 raft_consensus.cc:2781] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:10.082422 17302 raft_consensus.cc:482] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:10.082443 17302 raft_consensus.cc:3037] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:10.081966 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.077222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:43325 (local address 127.3.247.129:35181)
0806 06:53:10.077543 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.077553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:10.077695 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:10.077927 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:10.077928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:10.077929 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:10.078000 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.078002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.078864 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.078865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.080157 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:10.080160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.080965 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.080966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:10.081030 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.081034 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:10.081833 (+ 799us) client_negotiation.cc:240] Negotiation successful
0806 06:53:10.081877 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":39,"thread_start_us":114,"threads_started":1}
I20260806 06:53:10.083333 17302 raft_consensus.cc:504] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.083473 17302 leader_election.cc:288] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [CANDIDATE]: Term 1 election: Requested vote from peers b2a3c7f356be41dda5251b13491435bb (127.3.247.131:43325), 329e8ef85d354fcc87e49350440fd29a (127.3.247.130:37005)
I20260806 06:53:10.083636 17208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2f2fdca756449ba52c0753798b4caa" candidate_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a3c7f356be41dda5251b13491435bb" is_pre_election: true
I20260806 06:53:10.083783 17208 raft_consensus.cc:2454] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23132a4c62eb45bfb8b22e8c58fa91f3 in term 0.
I20260806 06:53:10.083992 17208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2f2fdca756449ba52c0753798b4caa" candidate_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a3c7f356be41dda5251b13491435bb"
I20260806 06:53:10.084064 17208 raft_consensus.cc:3037] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:10.084146 17081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2f2fdca756449ba52c0753798b4caa" candidate_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329e8ef85d354fcc87e49350440fd29a"
I20260806 06:53:10.084216 17081 raft_consensus.cc:3037] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:10.084872 17208 raft_consensus.cc:2454] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23132a4c62eb45bfb8b22e8c58fa91f3 in term 1.
I20260806 06:53:10.085047 16883 leader_election.cc:302] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23132a4c62eb45bfb8b22e8c58fa91f3, b2a3c7f356be41dda5251b13491435bb; no voters:
I20260806 06:53:10.085064 17081 raft_consensus.cc:2454] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23132a4c62eb45bfb8b22e8c58fa91f3 in term 1.
I20260806 06:53:10.085194 17302 raft_consensus.cc:2781] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:10.085340 17302 raft_consensus.cc:686] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [term 1 LEADER]: Becoming Leader. State: Replica: 23132a4c62eb45bfb8b22e8c58fa91f3, State: Running, Role: LEADER
I20260806 06:53:10.085427 17302 consensus_queue.cc:237] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } }
I20260806 06:53:10.086210 16794 catalog_manager.cc:5495] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23132a4c62eb45bfb8b22e8c58fa91f3 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23132a4c62eb45bfb8b22e8c58fa91f3" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 } health_report { overall_health: UNKNOWN } } }
I20260806 06:53:10.099753 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.096709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.3.247.129:43533)
0806 06:53:10.096750 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:10.096753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:10.096762 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:10.096807 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:10.096808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:10.096839 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:10.096921 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:10.097067 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.097772 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.098855 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:10.099150 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:10.099231 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:10.099392 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:10.099524 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:10.099611 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:10.099642 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:53:10.099664 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:53:10.101917 17081 raft_consensus.cc:1261] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 1 FOLLOWER]: Refusing update from remote peer 23132a4c62eb45bfb8b22e8c58fa91f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:10.101930 17208 raft_consensus.cc:1261] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 1 FOLLOWER]: Refusing update from remote peer 23132a4c62eb45bfb8b22e8c58fa91f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:10.102285 17306 consensus_queue.cc:1035] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a3c7f356be41dda5251b13491435bb" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:10.102468 17306 consensus_queue.cc:1035] T 4b2f2fdca756449ba52c0753798b4caa P 23132a4c62eb45bfb8b22e8c58fa91f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329e8ef85d354fcc87e49350440fd29a" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:10.105903 17309 mvcc.cc:205] Tried to move back new op lower bound from 7315452682654478336 to 7315452682590633984. Current Snapshot: MvccSnapshot[applied={T|T < 7315452682654478336}]
I20260806 06:53:10.106674 17310 mvcc.cc:205] Tried to move back new op lower bound from 7315452682654478336 to 7315452682590633984. Current Snapshot: MvccSnapshot[applied={T|T < 7315452682654478336}]
I20260806 06:53:10.107558 17311 mvcc.cc:205] Tried to move back new op lower bound from 7315452682654478336 to 7315452682590633984. Current Snapshot: MvccSnapshot[applied={T|T < 7315452682654478336}]
I20260806 06:53:10.108999 16794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:10.109103 16794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:10.110298 16794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d4c4d3d09244aa4a6c0c90a34eaa9cd: Sending DeleteTablet for 3 replicas of tablet 4b2f2fdca756449ba52c0753798b4caa
I20260806 06:53:10.110781 17195 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2f2fdca756449ba52c0753798b4caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:10 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:57726
I20260806 06:53:10.111011 17061 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2f2fdca756449ba52c0753798b4caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:10 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:47898
I20260806 06:53:10.111303 17319 tablet_replica.cc:330] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: stopping tablet replica
I20260806 06:53:10.111449 17319 raft_consensus.cc:2229] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:10.111431 16927 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2f2fdca756449ba52c0753798b4caa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:10 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:54574
I20260806 06:53:10.111635 17319 raft_consensus.cc:2258] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:10.111967 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16833
I20260806 06:53:10.112521 17318 tablet_replica.cc:330] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: stopping tablet replica
I20260806 06:53:10.112622 17318 raft_consensus.cc:2229] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:10.112720 17318 raft_consensus.cc:2258] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:10.113204 17318 ts_tablet_manager.cc:1905] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:10.115094 17318 ts_tablet_manager.cc:1918] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:10.115175 17318 log.cc:1196] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/4b2f2fdca756449ba52c0753798b4caa
I20260806 06:53:10.115443 17318 ts_tablet_manager.cc:1939] T 4b2f2fdca756449ba52c0753798b4caa P b2a3c7f356be41dda5251b13491435bb: Deleting consensus metadata
I20260806 06:53:10.115857 16782 catalog_manager.cc:4841] TS b2a3c7f356be41dda5251b13491435bb (127.3.247.131:43325): tablet 4b2f2fdca756449ba52c0753798b4caa (table test-table [id=5df83ee779e74432a47bf7ea81e140a2]) successfully deleted
I20260806 06:53:10.118755 17319 ts_tablet_manager.cc:1905] T 4b2f2fdca756449ba52c0753798b4caa P 329e8ef85d354fcc87e49350440fd29a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:53:10.118882 16780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:43533 (error 108)
I20260806 06:53:10.119452 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16997
I20260806 06:53:10.120646 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.120534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43533 (local address 127.0.0.1:54588)
0806 06:53:10.120581 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.120612 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:43533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:53:10.120746 16780 catalog_manager.cc:4584] TS 23132a4c62eb45bfb8b22e8c58fa91f3 (127.3.247.129:43533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b2f2fdca756449ba52c0753798b4caa: Network error: Client connection negotiation failed: client connection to 127.3.247.129:43533: connect: Connection refused (error 111)
I20260806 06:53:10.126628 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:10.126516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:37005 (local address 127.0.0.1:47906)
0806 06:53:10.126562 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:10.126584 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.130:37005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:53:10.126817 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17130
I20260806 06:53:10.133839 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 16762
I20260806 06:53:10.139569 4062 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:53:10.484761 4062 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:53:10.484882 4062 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:53:10.829704 4062 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:53:10.830 UTC [15829] LOG: received smart shutdown request
2026-08-06 06:53:10.833 UTC [15829] LOG: background worker "logical replication launcher" (PID 15838) exited with exit code 1
2026-08-06 06:53:10.834 UTC [15833] LOG: shutting down
2026-08-06 06:53:10.994 UTC [15829] LOG: database system is shut down
2026-08-06T06:53:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47614 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:53:11 dist-test-slave-0kls krb5kdc[17330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:53:11 dist-test-slave-0kls krb5kdc[17330](info): set up 2 sockets
Aug 06 06:53:11 dist-test-slave-0kls krb5kdc[17330](info): commencing operation
krb5kdc: starting...
W20260806 06:53:12.145900 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:53:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:53:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:53:12.266934 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33847
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:12.349634 17343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:12.349920 17343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:12.349963 17343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:12.351125 17343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:12.351197 17343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:12.351222 17343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:12.351253 17343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:12.351276 17343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:12.352825 17343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:12.353109 17343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:12.353334 17343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:12.358155 17351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:12.358114 17349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:12.358253 17348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:12.358315 17343 server_base.cc:1034] running on GCE node
I20260806 06:53:12.358640 17343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:12.358978 17343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:12.360158 17343 hybrid_clock.cc:648] HybridClock initialized: now 1785999192360134 us; error 40 us; skew 500 ppm
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:12.362710 17343 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:12.363166 17343 webserver.cc:460] Webserver started at http://127.3.247.190:45003/ using document root <none> and password file <none>
I20260806 06:53:12.363366 17343 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:12.363408 17343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:12.363495 17343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:12.364500 17343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "d742a4ed24c646aba89efc9bbd023561"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "e287c980a0c9b7be2d921abd5c9b7e95"
server_key_iv: "8e7b9a681858945e468f0b7ba2722c6a"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.364790 17343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "d742a4ed24c646aba89efc9bbd023561"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "e287c980a0c9b7be2d921abd5c9b7e95"
server_key_iv: "8e7b9a681858945e468f0b7ba2722c6a"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.366087 17343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:12.366770 17358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:12.366966 17343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:12.367048 17343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "d742a4ed24c646aba89efc9bbd023561"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "e287c980a0c9b7be2d921abd5c9b7e95"
server_key_iv: "8e7b9a681858945e468f0b7ba2722c6a"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.367103 17343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:12.384105 17343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:12.386013 17343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:12.386132 17343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:12.390089 17343 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:33847
I20260806 06:53:12.390164 17410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:33847 every 8 connection(s)
I20260806 06:53:12.390503 17343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:12.391362 17411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:12.392163 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17343
I20260806 06:53:12.392256 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:53:12.392491 4062 external_mini_cluster.cc:1428] Setting key c8ade3aa8ae39d9407b8309776b154bf
I20260806 06:53:12.393121 17411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.189@KRBTEST.COM" created.
I20260806 06:53:12.404642 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.393875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.3.247.190:33847)
0806 06:53:12.394103 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.394119 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.394321 (+ 202us) server_negotiation.cc:407] Connection header received
0806 06:53:12.395070 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.395072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.395089 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.395159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.395950 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.396751 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.398217 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.398528 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.398620 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.401005 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.401014 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.401014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.401023 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.402943 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.403404 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.403406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.403407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.403494 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.403710 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.403711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.403712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.403961 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.404059 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.404236 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.404393 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":174,"spinlock_wait_cycles":7424,"thread_start_us":92,"threads_started":1}
I20260806 06:53:12.404641 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.393875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33847 (local address 127.0.0.1:52302)
0806 06:53:12.394108 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.394179 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.394899 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.395211 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.395214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.395617 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.395847 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.395853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.396854 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.396856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.398111 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.398114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.398627 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.398628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.398718 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.399264 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.399275 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.400909 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.403080 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.403088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.403091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.403331 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.403565 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.403565 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.403566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.403639 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.404039 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.404044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.404113 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.404211 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.404445 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":95,"thread_start_us":90,"threads_started":1}
I20260806 06:53:12.405519 17411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:53:12.405959 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.405767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50924)
0806 06:53:12.405877 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.405936 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 06:53:12.406132 17362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
I20260806 06:53:12.406571 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.406476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50930)
0806 06:53:12.406524 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.406548 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:53:12.406678 17411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:39295: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
I20260806 06:53:12.421204 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:39295
--webserver_interface=127.3.247.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:12.455906 17411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } attempt: 1
I20260806 06:53:12.456439 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.456267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50942)
0806 06:53:12.456347 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.456401 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:53:12.456820 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.456722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50958)
0806 06:53:12.456769 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.456798 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:53:12.456924 17411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:39295: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
W20260806 06:53:12.503077 17418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:12.503278 17418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:12.503300 17418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:12.504338 17418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:12.504369 17418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:12.504383 17418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:12.504396 17418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:12.504408 17418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:12.505914 17418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.247.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:12.506094 17418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:12.506253 17418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:12.510823 17423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:12.510987 17418 server_base.cc:1034] running on GCE node
W20260806 06:53:12.510864 17426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:12.510823 17424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:12.511338 17418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:12.511605 17418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:12.512776 17418 hybrid_clock.cc:648] HybridClock initialized: now 1785999192512753 us; error 36 us; skew 500 ppm
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:12.515209 17418 init.cc:377] Logged in from keytab as kudu/127.3.247.189@KRBTEST.COM (short username kudu)
I20260806 06:53:12.515664 17418 webserver.cc:460] Webserver started at http://127.3.247.189:46823/ using document root <none> and password file <none>
I20260806 06:53:12.515915 17418 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:12.515985 17418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:12.516117 17418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:12.517148 17418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data/instance:
uuid: "3b488392fc9f48c1b125b374dff3f5bb"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "9785da177c3a039060cd4ff661030c18"
server_key_iv: "cd821de314a62caf64623a2692675bbc"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.517509 17418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal/instance:
uuid: "3b488392fc9f48c1b125b374dff3f5bb"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "9785da177c3a039060cd4ff661030c18"
server_key_iv: "cd821de314a62caf64623a2692675bbc"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.518870 17418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:12.519570 17433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:12.519748 17418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:12.519855 17418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "3b488392fc9f48c1b125b374dff3f5bb"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "9785da177c3a039060cd4ff661030c18"
server_key_iv: "cd821de314a62caf64623a2692675bbc"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.519941 17418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:12.525070 17411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } attempt: 2
I20260806 06:53:12.525429 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.525309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50968)
0806 06:53:12.525364 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.525400 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:53:12.525764 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.525685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50980)
0806 06:53:12.525724 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.525747 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:53:12.525897 17411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:39295: Network error: Client connection negotiation failed: client connection to 127.3.247.189:39295: connect: Connection refused (error 111)
I20260806 06:53:12.543452 17418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:12.545370 17418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:12.545481 17418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:12.549645 17418 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.189:39295
I20260806 06:53:12.549842 17485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.189:39295 every 8 connection(s)
I20260806 06:53:12.550122 17418 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
I20260806 06:53:12.550961 17486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:12.552497 17486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:53:12.556414 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17418
I20260806 06:53:12.556543 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-1/wal/instance
I20260806 06:53:12.556753 4062 external_mini_cluster.cc:1428] Setting key bdaff03d561029ba4ae765dc4b292632
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:12.562191 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.553203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.3.247.190:33847)
0806 06:53:12.553274 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.553276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.553437 (+ 161us) server_negotiation.cc:407] Connection header received
0806 06:53:12.553935 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.553936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.553947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.554009 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.554568 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.555116 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.556041 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.556231 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.556284 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.558780 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.558782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.558783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.558792 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.560555 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.561043 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.561046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.561046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.561115 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.561357 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.561358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.561358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.561518 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.561596 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.561943 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.562045 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:53:12.562222 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.553150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33847 (local address 127.0.0.1:52308)
0806 06:53:12.553303 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.553346 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.553838 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.554062 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.554065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.554295 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.554485 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.554489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.555198 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.555199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.555966 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.555968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.556312 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.556313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.556464 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.556996 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.557006 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.558678 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.560647 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.560654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.560656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.560961 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.561187 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.561187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.561187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.561286 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.561609 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.561615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.561752 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.561910 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.562052 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260806 06:53:12.562870 17486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.247.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.188@KRBTEST.COM" created.
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:53:12.571902 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.563154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50984)
0806 06:53:12.563231 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.563241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.563287 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.563626 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.563627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.563759 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.563816 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.563818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.565179 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.565180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.565761 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.565763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.566300 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.566301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.566363 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.566827 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.566831 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.568870 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.570695 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.570696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.570696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.570980 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.571212 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.571213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.571213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.571270 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.571551 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.571552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.571595 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.571681 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.571789 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 06:53:12.571992 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.563234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.3.247.189:39295)
0806 06:53:12.563437 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.563443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.563468 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:53:12.563490 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.563492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.563505 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.563589 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.563913 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.565079 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.565911 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.566221 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.566280 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.568951 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.568959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.568959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.568966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.570614 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.571087 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.571090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.571091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.571147 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.571334 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.571334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.571335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.571474 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.571562 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.571713 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.571860 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260806 06:53:12.572882 17486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:53:12.573216 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.573114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39180)
0806 06:53:12.573157 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.573199 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:53:12.573342 17436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
I20260806 06:53:12.573839 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.573706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39194)
0806 06:53:12.573771 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.573815 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 06:53:12.573948 17486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43137: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
I20260806 06:53:12.585379 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:43137
--webserver_interface=127.3.247.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:12.617079 17411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } attempt: 3
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:53:12.623154 17486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } attempt: 1
I20260806 06:53:12.623608 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.623450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39196)
0806 06:53:12.623543 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.623571 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 06:53:12.624073 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.623977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39202)
0806 06:53:12.624014 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.624043 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 06:53:12.624640 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.617596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.3.247.189:39295)
0806 06:53:12.617671 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.617673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.617684 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:53:12.617737 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.617738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.617749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.617870 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.618202 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.618719 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.619438 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.619658 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.619737 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.621651 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.621653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.621653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.621663 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.623416 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.623828 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.623829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.623829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.623893 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.624099 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.624101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.624101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.624243 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.624342 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.624476 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.624553 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:53:12.624648 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.617518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50986)
0806 06:53:12.617582 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.617598 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.617673 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.617881 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.617882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.618052 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.618110 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.618112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.618825 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.618826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.619348 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.619350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.619735 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.619736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.619825 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.620334 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.620340 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.621544 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.623507 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.623509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.623509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.623742 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.623959 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.623960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.623961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.624034 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.624334 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.624335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.624386 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.624453 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.624552 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:53:12.624783 17486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43137: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
I20260806 06:53:12.625118 17411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:53:12.625429 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.625341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39214)
0806 06:53:12.625381 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.625410 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:53:12.625751 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.625657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39218)
0806 06:53:12.625704 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.625728 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:53:12.625882 17411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43137: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
I20260806 06:53:12.657078 17411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } attempt: 1
I20260806 06:53:12.657663 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.657483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39224)
0806 06:53:12.657575 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.657625 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 06:53:12.658119 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.658014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39236)
0806 06:53:12.658065 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.658094 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:53:12.658233 17411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43137: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
W20260806 06:53:12.669224 17493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:12.669445 17493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:12.669468 17493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:12.670817 17493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:12.670867 17493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:12.670882 17493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:12.670902 17493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:12.670914 17493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:12.672421 17493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33847,127.3.247.189:39295,127.3.247.188:43137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:43137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.247.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:12.672621 17493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:12.672798 17493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:12.677624 17499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:12.677695 17493 server_base.cc:1034] running on GCE node
W20260806 06:53:12.677628 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:12.677639 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:12.678097 17493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:12.678401 17493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:12.679574 17493 hybrid_clock.cc:648] HybridClock initialized: now 1785999192679545 us; error 41 us; skew 500 ppm
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:12.682106 17493 init.cc:377] Logged in from keytab as kudu/127.3.247.188@KRBTEST.COM (short username kudu)
I20260806 06:53:12.682601 17493 webserver.cc:460] Webserver started at http://127.3.247.188:39067/ using document root <none> and password file <none>
I20260806 06:53:12.682868 17493 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:12.682917 17493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:12.683063 17493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:12.684232 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data/instance:
uuid: "4b0a74838faa4a69ae8bac2d47c7dd21"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "23c9620024066e9d513754359bb681c6"
server_key_iv: "1bc686112a7c7f005dcb62786eb47095"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.684576 17493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal/instance:
uuid: "4b0a74838faa4a69ae8bac2d47c7dd21"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "23c9620024066e9d513754359bb681c6"
server_key_iv: "1bc686112a7c7f005dcb62786eb47095"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.685985 17493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:53:12.686697 17508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:12.686896 17493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:12.687000 17493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "4b0a74838faa4a69ae8bac2d47c7dd21"
format_stamp: "Formatted at 2026-08-06 06:53:12 on dist-test-slave-0kls"
server_key: "23c9620024066e9d513754359bb681c6"
server_key_iv: "1bc686112a7c7f005dcb62786eb47095"
server_key_version: "encryptionkey@0"
I20260806 06:53:12.687069 17493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:12.692919 17486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } attempt: 2
I20260806 06:53:12.693321 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.693191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39246)
0806 06:53:12.693252 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.693289 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:53:12.693754 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.693620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39248)
0806 06:53:12.693696 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.693722 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 06:53:12.693931 17486 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43137: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43137: connect: Connection refused (error 111)
I20260806 06:53:12.694762 17493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:12.696632 17493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:12.696791 17493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:12.700532 17493 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.188:43137
I20260806 06:53:12.700567 17560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.188:43137 every 8 connection(s)
I20260806 06:53:12.700913 17493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
I20260806 06:53:12.701864 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:12.703397 17561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:12.710345 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17493
I20260806 06:53:12.710484 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785999040366271-4062-0/minicluster-data/master-2/wal/instance
I20260806 06:53:12.710704 4062 external_mini_cluster.cc:1428] Setting key 09e3482a0e2c44b77b1d7e1fb19cabec
I20260806 06:53:12.712574 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.704133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.3.247.190:33847)
0806 06:53:12.704208 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.704211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.704416 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:53:12.704887 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.704888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.704900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.704954 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.705562 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.706083 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.707064 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.707255 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.707342 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.709370 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.709371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.709372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.709379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.711119 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.711572 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.711573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.711573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.711642 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.711906 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.711907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.711907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.712033 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.712116 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.712392 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.712477 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:53:12.712788 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.704097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33847 (local address 127.0.0.1:52310)
0806 06:53:12.704292 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.704330 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.704792 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.705023 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.705026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.705291 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.705487 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.705491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.706189 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.706190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.706985 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.706987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.707354 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.707355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.707465 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.707923 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.707931 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.709279 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.711226 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.711231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.711234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.711469 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.711718 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.711719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.711720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.711830 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.712108 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.712114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.712255 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.712370 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.712525 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260806 06:53:12.713348 17561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:53:12.720431 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.713728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.3.247.189:39295)
0806 06:53:12.713774 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.713776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.713802 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:53:12.713852 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.713853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.713863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.713923 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.714195 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.714630 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.715326 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.715637 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.715725 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.717520 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.717522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.717522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.717530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.719135 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719518 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.719520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.719520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.719579 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719806 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.719808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.719808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.719975 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.720060 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.720234 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.720331 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:53:12.720441 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.713661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:39295 (local address 127.0.0.1:50994)
0806 06:53:12.713715 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.713727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.713763 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.713925 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.713926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.714064 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.714109 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.714111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.714703 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.714704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.715237 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.715239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.715718 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.715719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.715767 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.716151 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.716157 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.717407 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.719223 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.719225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.719446 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.719650 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.719651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.719710 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.720073 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.720074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.720117 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.720193 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.720290 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:53:12.720468 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.711101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.3.247.188:43137)
0806 06:53:12.711371 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.711375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.711393 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:53:12.711410 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.711412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.711422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.711502 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.712045 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.712826 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.713789 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.714275 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.714352 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.716939 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.716948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.716948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.716955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.719019 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719430 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.719433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.719433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.719517 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.719818 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.719819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.719819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.720010 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.720154 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.720228 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.720355 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":244,"thread_start_us":95,"threads_started":1}
I20260806 06:53:12.721339 17561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:12.729000 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.721704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39272 (local address 127.3.247.188:43137)
0806 06:53:12.721745 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.721747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.721773 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:53:12.721814 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.721815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.721824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.721885 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.722157 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.722858 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.723795 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.724103 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.724176 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.726064 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.726066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.726066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.726073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.727807 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.728184 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.728184 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.728185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.728231 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.728456 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.728457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.728457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.728582 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.728651 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.728797 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.728908 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":22}
I20260806 06:53:12.729070 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.721646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39272)
0806 06:53:12.721686 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.721694 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.721731 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.721892 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.721893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.722046 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.722081 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.722083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.722932 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.722933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.723690 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.723692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.724185 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.724186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.724242 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.724661 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.724665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.725978 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.727900 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.727901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.727902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.728105 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.728310 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.728311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.728312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.728377 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.728660 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.728660 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.728697 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.728752 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.728845 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:53:12.729928 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.721277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.3.247.190:33847)
0806 06:53:12.721321 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.721323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.721331 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:12.721355 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.721356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.721366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.721421 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.722055 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.722836 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.723807 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.724103 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.724176 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.726750 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.726751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.726752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.726758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.728500 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.728919 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.728920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.728921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.728987 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.729208 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.729209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.729209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.729308 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.729535 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.729725 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.729825 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:53:12.731364 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: Bootstrap starting.
I20260806 06:53:12.732403 17561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:12.732769 17561 log.cc:824] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:12.733358 17411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } attempt: 2
I20260806 06:53:12.734288 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: No bootstrap required, opened a new log
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:53:12.737330 17561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:12.737599 17561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:12.737694 17561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0a74838faa4a69ae8bac2d47c7dd21, State: Initialized, Role: FOLLOWER
I20260806 06:53:12.737939 17561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:53:12.738348 17561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:12.738384 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:12.738579 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:12.739342 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.730742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.3.247.189:39295)
0806 06:53:12.730788 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.730789 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.730796 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:53:12.730817 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.730818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.730827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.730893 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.731360 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.731967 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.732737 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.733169 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.733270 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.736065 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.736067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.736068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.736075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.737888 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.738294 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.738296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.738296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.738356 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.738595 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.738596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.738596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.738727 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.738906 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.739157 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.739249 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:53:12.740825 17578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:53:12.740945 17578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:53:12.741195 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:53:12.742545 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.733714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39282)
0806 06:53:12.733769 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.733828 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.733869 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.734060 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.734061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.734214 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.734250 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.734252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.734894 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.734895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.735585 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.735588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.736090 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.736091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.736149 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.736816 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.736821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.738707 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.741385 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.741386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.741386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.741575 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.741841 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.741842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.741842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.741905 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.742184 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.742185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.742234 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.742317 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.742422 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:53:12.742858 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.733760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.3.247.188:43137)
0806 06:53:12.733854 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.733856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.733865 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:12.733960 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.733961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.733977 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.734040 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.734331 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.734806 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.735707 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.735985 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.738801 (+ 2816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.738876 (+ 75us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.738877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.738877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.738882 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.741301 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.741660 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.741661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.741662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.741720 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.741975 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.741976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.741976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.742113 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.742193 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.742685 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.742768 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:53:12.741104 17493 master_runner.cc:186] Error getting master registration for 127.3.247.190:33847: OK, instance_id { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" instance_seqno: 1785999192388898 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:53:12.744285 17493 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:53:12.745482 17411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: Bootstrap starting.
I20260806 06:53:12.746327 17411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:12.746697 17411 log.cc:824] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:12.747577 17411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: No bootstrap required, opened a new log
I20260806 06:53:12.749385 17411 raft_consensus.cc:348] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:12.749563 17411 raft_consensus.cc:374] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:12.749655 17411 raft_consensus.cc:729] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d742a4ed24c646aba89efc9bbd023561, State: Initialized, Role: FOLLOWER
I20260806 06:53:12.749889 17411 consensus_queue.cc:260] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:12.750335 17411 sys_catalog.cc:564] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 06:53:12.752491 17599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:53:12.752594 17599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:53:12.755015 17490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.742749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.3.247.189:39295)
0806 06:53:12.742788 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.742790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.746503 (+ 3713us) server_negotiation.cc:407] Connection header received
0806 06:53:12.746564 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.746565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.746577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.746655 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.747372 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.748061 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.749085 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.749384 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.749466 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.751304 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.751305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.751305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.751313 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.753088 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.753512 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.753513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.753513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.753572 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.753812 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.753813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.753813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.753941 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.754592 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.754835 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.754924 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:53:12.753475 17343 master_runner.cc:186] Error getting master registration for 127.3.247.189:39295: OK, instance_id { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" instance_seqno: 1785999192548401 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:53:12.756902 17343 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:53:12.758878 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.746103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.3.247.188:43137)
0806 06:53:12.746143 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.746145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.750074 (+ 3929us) server_negotiation.cc:407] Connection header received
0806 06:53:12.750104 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.750105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.750118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.750194 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.751070 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.752060 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.753289 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.753571 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.753622 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.755906 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.755907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.755907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.755915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.757600 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.758027 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.758028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.758029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.758088 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.758285 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.758286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.758286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.758424 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.758508 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.758706 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.758785 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:53:12.759196 17588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:12.759357 17588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:12.762984 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.746322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.3.247.190:33847)
0806 06:53:12.746363 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.746365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.750231 (+ 3866us) server_negotiation.cc:407] Connection header received
0806 06:53:12.750257 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.750258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.750269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.750322 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.755937 (+ 5615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.757130 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.758077 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.758354 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.758401 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.760017 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.760018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.760019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.760026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.761643 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.762097 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.762098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.762098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.762159 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.762390 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.762391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.762392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.762547 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.762654 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.762816 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.762895 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:53:12.785096 17486 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } attempt: 3
Aug 06 06:53:12 dist-test-slave-0kls krb5kdc[17330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999192, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:53:12.793757 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.785529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.3.247.188:43137)
0806 06:53:12.785575 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:12.785577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:12.785588 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:53:12.785658 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:12.785659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:12.785670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:12.785739 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:12.786137 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.786917 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.787946 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.788244 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.788321 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.790680 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:12.790682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:12.790682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:12.790690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:12.792448 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.792895 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.792896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.792896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.792959 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:12.793188 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:12.793189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:12.793189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:12.793327 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:12.793396 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:12.793580 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:53:12.793657 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:53:12.793764 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:12.785464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43137 (local address 127.0.0.1:39302)
0806 06:53:12.785523 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:12.785536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:12.785595 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:12.785761 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:12.785762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:12.785987 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:12.786047 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.786049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.787023 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.787025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.787859 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:12.787862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:12.788345 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:12.788346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:12.788414 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:12.788932 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:12.788937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:12.790594 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:12.792540 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.792541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.792542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.792789 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.793037 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:12.793038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:12.793038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:12.793113 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:12.793401 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:12.793402 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:12.793462 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 06:53:12.793544 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:53:12.793651 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:53:12.795768 17486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: Bootstrap starting.
I20260806 06:53:12.796600 17486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:12.796880 17486 log.cc:824] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:12.797821 17486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: No bootstrap required, opened a new log
I20260806 06:53:12.799840 17486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:12.799983 17486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:12.800014 17486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b488392fc9f48c1b125b374dff3f5bb, State: Initialized, Role: FOLLOWER
I20260806 06:53:12.800263 17486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:12.800673 17486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:12.800662 17602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:12.800917 17602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:53:12.803951 17613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:53:12.804214 17613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:53:12.805066 17418 master_runner.cc:263] No leader master found from master 3b488392fc9f48c1b125b374dff3f5bb
I20260806 06:53:12.805120 17418 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:53:12.886978 17364 tablet.cc:2369] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:53:12.947369 17514 tablet.cc:2369] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:13.041608 17588 raft_consensus.cc:482] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:13.041736 17588 raft_consensus.cc:504] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:13.042070 17588 leader_election.cc:288] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b488392fc9f48c1b125b374dff3f5bb (127.3.247.189:39295), 4b0a74838faa4a69ae8bac2d47c7dd21 (127.3.247.188:43137)
I20260806 06:53:13.042376 17535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d742a4ed24c646aba89efc9bbd023561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" is_pre_election: true
I20260806 06:53:13.042418 17460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d742a4ed24c646aba89efc9bbd023561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b488392fc9f48c1b125b374dff3f5bb" is_pre_election: true
I20260806 06:53:13.042572 17535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d742a4ed24c646aba89efc9bbd023561 in term 0.
I20260806 06:53:13.042569 17460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d742a4ed24c646aba89efc9bbd023561 in term 0.
I20260806 06:53:13.042809 17361 leader_election.cc:302] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0a74838faa4a69ae8bac2d47c7dd21, d742a4ed24c646aba89efc9bbd023561; no voters:
I20260806 06:53:13.042953 17588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:13.043012 17588 raft_consensus.cc:482] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:13.043035 17588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:13.043782 17588 raft_consensus.cc:504] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:13.043939 17588 leader_election.cc:288] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [CANDIDATE]: Term 1 election: Requested vote from peers 3b488392fc9f48c1b125b374dff3f5bb (127.3.247.189:39295), 4b0a74838faa4a69ae8bac2d47c7dd21 (127.3.247.188:43137)
I20260806 06:53:13.044082 17460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d742a4ed24c646aba89efc9bbd023561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b488392fc9f48c1b125b374dff3f5bb"
I20260806 06:53:13.044085 17535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d742a4ed24c646aba89efc9bbd023561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21"
I20260806 06:53:13.044168 17460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:13.044168 17535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:13.045011 17460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d742a4ed24c646aba89efc9bbd023561 in term 1.
I20260806 06:53:13.045011 17535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d742a4ed24c646aba89efc9bbd023561 in term 1.
I20260806 06:53:13.045223 17362 leader_election.cc:302] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b488392fc9f48c1b125b374dff3f5bb, d742a4ed24c646aba89efc9bbd023561; no voters:
I20260806 06:53:13.045341 17588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:13.045521 17588 raft_consensus.cc:686] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [term 1 LEADER]: Becoming Leader. State: Replica: d742a4ed24c646aba89efc9bbd023561, State: Running, Role: LEADER
I20260806 06:53:13.045627 17588 consensus_queue.cc:237] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } }
I20260806 06:53:13.046072 17615 sys_catalog.cc:455] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d742a4ed24c646aba89efc9bbd023561. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.046162 17615 sys_catalog.cc:458] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: This master's current role is: LEADER
W20260806 06:53:13.046224 17439 tablet.cc:2369] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:13.046427 17617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:13.046612 17617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:13.047978 17460 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [term 1 FOLLOWER]: Refusing update from remote peer d742a4ed24c646aba89efc9bbd023561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:13.047971 17535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [term 1 FOLLOWER]: Refusing update from remote peer d742a4ed24c646aba89efc9bbd023561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:13.048266 17615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:13.048413 17588 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:13.049614 17602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d742a4ed24c646aba89efc9bbd023561. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.049712 17602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:13.049685 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d742a4ed24c646aba89efc9bbd023561. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.049770 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:13.050629 17615 sys_catalog.cc:455] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.050654 17623 sys_catalog.cc:455] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.050733 17615 sys_catalog.cc:458] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:13.050738 17623 sys_catalog.cc:458] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:13.051048 17617 catalog_manager.cc:1347] Generated new cluster ID: fac8d5b2cd0941f78650c3a3f80feaec
I20260806 06:53:13.051093 17617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:53:13.051256 17602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.051321 17602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:13.051383 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d742a4ed24c646aba89efc9bbd023561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d742a4ed24c646aba89efc9bbd023561" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33847 } } peers { permanent_uuid: "3b488392fc9f48c1b125b374dff3f5bb" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 39295 } } peers { permanent_uuid: "4b0a74838faa4a69ae8bac2d47c7dd21" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43137 } } }
I20260806 06:53:13.051440 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:53:13.051478 17620 mvcc.cc:205] Tried to move back new op lower bound from 7315452694722162688 to 7315452694715670528. Current Snapshot: MvccSnapshot[applied={T|T < 7315452694722162688 or (T in {7315452694722162688})}]
I20260806 06:53:13.054064 17619 mvcc.cc:205] Tried to move back new op lower bound from 7315452694722162688 to 7315452694715670528. Current Snapshot: MvccSnapshot[applied={T|T < 7315452694722162688 or (T in {7315452694722162688})}]
I20260806 06:53:13.065220 17617 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:53:13.065949 17617 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:13.070007 17617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d742a4ed24c646aba89efc9bbd023561: Generated new TSK 0
I20260806 06:53:13.070314 17617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:53:13.741628 17578 catalog_manager.cc:1259] Loaded cluster ID: fac8d5b2cd0941f78650c3a3f80feaec
I20260806 06:53:13.741741 17578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: loading cluster ID for follower catalog manager: success
I20260806 06:53:13.742703 17578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: acquiring CA information for follower catalog manager: success
I20260806 06:53:13.743064 17578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4b0a74838faa4a69ae8bac2d47c7dd21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:53:13.804828 17613 catalog_manager.cc:1259] Loaded cluster ID: fac8d5b2cd0941f78650c3a3f80feaec
I20260806 06:53:13.804922 17613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: loading cluster ID for follower catalog manager: success
I20260806 06:53:13.806190 17613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: acquiring CA information for follower catalog manager: success
I20260806 06:53:13.806614 17613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b488392fc9f48c1b125b374dff3f5bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:53:14.127758 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17343
I20260806 06:53:14.133332 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17418
I20260806 06:53:14.138698 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17493
2026-08-06T06:53:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3033 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:53:14 dist-test-slave-0kls krb5kdc[17632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:53:14 dist-test-slave-0kls krb5kdc[17632](info): set up 2 sockets
Aug 06 06:53:14 dist-test-slave-0kls krb5kdc[17632](info): commencing operation
krb5kdc: starting...
W20260806 06:53:15.181955 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:53:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:53:15.304203 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39499
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:15.387413 17645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:15.387626 17645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:15.387648 17645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:15.388891 17645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:15.388933 17645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:15.388955 17645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:15.388973 17645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:15.388993 17645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:15.390614 17645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:15.390898 17645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:15.391142 17645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:15.396215 17651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:15.396126 17650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:15.396174 17653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:15.396381 17645 server_base.cc:1034] running on GCE node
I20260806 06:53:15.396662 17645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:15.397047 17645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:15.398232 17645 hybrid_clock.cc:648] HybridClock initialized: now 1785999195398211 us; error 47 us; skew 500 ppm
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999195, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:15.400842 17645 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:15.401366 17645 webserver.cc:460] Webserver started at http://127.3.247.190:33483/ using document root <none> and password file <none>
I20260806 06:53:15.401608 17645 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:15.401695 17645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:15.402025 17645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:15.403260 17645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "469d7b079ed745eab0c07528348f3fc8"
format_stamp: "Formatted at 2026-08-06 06:53:15 on dist-test-slave-0kls"
server_key: "17cf240e2e73df5f406c13b5c555f81b"
server_key_iv: "11549bac0c124ff0c63368de338591c3"
server_key_version: "encryptionkey@0"
I20260806 06:53:15.403805 17645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "469d7b079ed745eab0c07528348f3fc8"
format_stamp: "Formatted at 2026-08-06 06:53:15 on dist-test-slave-0kls"
server_key: "17cf240e2e73df5f406c13b5c555f81b"
server_key_iv: "11549bac0c124ff0c63368de338591c3"
server_key_version: "encryptionkey@0"
I20260806 06:53:15.405524 17645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:15.406353 17660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:15.406546 17645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:15.406646 17645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "469d7b079ed745eab0c07528348f3fc8"
format_stamp: "Formatted at 2026-08-06 06:53:15 on dist-test-slave-0kls"
server_key: "17cf240e2e73df5f406c13b5c555f81b"
server_key_iv: "11549bac0c124ff0c63368de338591c3"
server_key_version: "encryptionkey@0"
I20260806 06:53:15.406729 17645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:15.420987 17645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:15.423017 17645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:15.423188 17645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:15.427393 17645 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39499
I20260806 06:53:15.427448 17712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39499 every 8 connection(s)
I20260806 06:53:15.427836 17645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:15.428675 17713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:15.429402 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17645
I20260806 06:53:15.429522 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:53:15.429697 4062 external_mini_cluster.cc:1428] Setting key 3de50e240459f5756a46399fef7fd231
I20260806 06:53:15.431766 17713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: Bootstrap starting.
I20260806 06:53:15.432514 17713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:15.432829 17713 log.cc:824] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:15.433569 17713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: No bootstrap required, opened a new log
I20260806 06:53:15.436023 17713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } }
I20260806 06:53:15.436219 17713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:15.436252 17713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469d7b079ed745eab0c07528348f3fc8, State: Initialized, Role: FOLLOWER
I20260806 06:53:15.436398 17713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } }
I20260806 06:53:15.436496 17713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:15.436540 17713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:15.436596 17713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:15.437206 17713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } }
I20260806 06:53:15.437327 17713 leader_election.cc:302] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 469d7b079ed745eab0c07528348f3fc8; no voters:
I20260806 06:53:15.437541 17713 leader_election.cc:288] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:53:15.437569 17718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:15.437760 17718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [term 1 LEADER]: Becoming Leader. State: Replica: 469d7b079ed745eab0c07528348f3fc8, State: Running, Role: LEADER
I20260806 06:53:15.437886 17713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:15.437929 17718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } }
I20260806 06:53:15.438336 17718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469d7b079ed745eab0c07528348f3fc8. Latest consensus state: current_term: 1 leader_uuid: "469d7b079ed745eab0c07528348f3fc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } } }
I20260806 06:53:15.438778 17718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:15.438658 17719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "469d7b079ed745eab0c07528348f3fc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469d7b079ed745eab0c07528348f3fc8" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39499 } } }
I20260806 06:53:15.439239 17719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8 [sys.catalog]: This master's current role is: LEADER
W20260806 06:53:15.439710 17734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:53:15.439764 17734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:53:15.439813 17728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:15.439990 17728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:15.441576 17728 catalog_manager.cc:1347] Generated new cluster ID: 21b61ac20d014e3482cfd3ee83714e99
I20260806 06:53:15.441623 17728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:53:15.441776 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:15.430230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.3.247.190:39499)
0806 06:53:15.430424 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:15.430435 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:15.430468 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:53:15.431019 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:15.431029 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:15.431103 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:15.431254 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:15.431845 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:15.433262 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:15.434348 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:15.434758 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:15.434844 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:15.437641 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:15.437646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:15.437646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:15.437658 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:15.439905 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:15.440307 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:15.440310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:15.440310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:15.440423 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:15.440643 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:15.440645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:15.440645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:15.440902 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:15.441059 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:15.441443 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 06:53:15.441587 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260806 06:53:15.442672 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:53:15.453393 17728 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:53:15.454201 17728 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:15.463140 17728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 469d7b079ed745eab0c07528348f3fc8: Generated new TSK 0
I20260806 06:53:15.463366 17728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:53:15.496853 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17645
2026-08-06T06:53:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1356 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17740](info): set up 2 sockets
Aug 06 06:53:15 dist-test-slave-0kls krb5kdc[17740](info): commencing operation
krb5kdc: starting...
W20260806 06:53:16.538033 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:53:16 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:53:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:53:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:53:16.656167 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:16.740011 17753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:16.740285 17753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:16.740325 17753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:16.741622 17753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:16.741703 17753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:16.741729 17753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:16.741760 17753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:16.741840 17753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:16.743486 17753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:16.743783 17753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:16.744030 17753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:16.748945 17759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:16.749118 17753 server_base.cc:1034] running on GCE node
W20260806 06:53:16.749056 17758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:16.749059 17761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:16.749572 17753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:16.749940 17753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:16.751062 17753 hybrid_clock.cc:648] HybridClock initialized: now 1785999196751039 us; error 36 us; skew 500 ppm
Aug 06 06:53:16 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999196, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:16.754411 17753 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:16.754889 17753 webserver.cc:460] Webserver started at http://127.3.247.190:45717/ using document root <none> and password file <none>
I20260806 06:53:16.755143 17753 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:16.755214 17753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:16.755384 17753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:16.756464 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "a50d9607832b4152a0b5d6d492868a52"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "a8b644883b5f499edf04a7124d32f51d"
server_key_iv: "868ee0cdcb88987770d14c1011d4daa7"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.756834 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "a50d9607832b4152a0b5d6d492868a52"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "a8b644883b5f499edf04a7124d32f51d"
server_key_iv: "868ee0cdcb88987770d14c1011d4daa7"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.758240 17753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:53:16.758957 17768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:16.759128 17753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:53:16.759222 17753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "a50d9607832b4152a0b5d6d492868a52"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "a8b644883b5f499edf04a7124d32f51d"
server_key_iv: "868ee0cdcb88987770d14c1011d4daa7"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.759308 17753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:16.786701 17753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:16.788792 17753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:16.788969 17753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:16.792887 17753 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38181
I20260806 06:53:16.792953 17820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38181 every 8 connection(s)
I20260806 06:53:16.793334 17753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:16.794449 17821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:16.796800 17821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap starting.
I20260806 06:53:16.797425 17821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:16.797725 17821 log.cc:824] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:16.798491 17821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: No bootstrap required, opened a new log
I20260806 06:53:16.800052 17821 raft_consensus.cc:348] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:16.800190 17821 raft_consensus.cc:374] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:16.800213 17821 raft_consensus.cc:729] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Initialized, Role: FOLLOWER
I20260806 06:53:16.800354 17821 consensus_queue.cc:260] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:16.800437 17821 raft_consensus.cc:388] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:16.800462 17821 raft_consensus.cc:482] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:16.800490 17821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:16.801296 17821 raft_consensus.cc:504] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:16.801383 17821 leader_election.cc:302] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a50d9607832b4152a0b5d6d492868a52; no voters:
I20260806 06:53:16.801507 17821 leader_election.cc:288] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:53:16.801554 17824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:16.801738 17824 raft_consensus.cc:686] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 LEADER]: Becoming Leader. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Running, Role: LEADER
I20260806 06:53:16.801831 17821 sys_catalog.cc:564] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:16.801870 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17753
I20260806 06:53:16.801905 17824 consensus_queue.cc:237] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:16.801992 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:53:16.802205 4062 external_mini_cluster.cc:1428] Setting key 829c6ea2117563b4f52e8d386718df37
I20260806 06:53:16.802289 17824 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a50d9607832b4152a0b5d6d492868a52. Latest consensus state: current_term: 1 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:16.802354 17824 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:16.802486 17826 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:16.802538 17826 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:16.802641 17830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:16.803376 17830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:16.805155 17830 catalog_manager.cc:1347] Generated new cluster ID: 074c096220294ece862d59fdb2c75878
I20260806 06:53:16.805209 17830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:53:16 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:16.811929 17830 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:53:16.812666 17830 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:16.816432 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:16.802893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.3.247.190:38181)
0806 06:53:16.803044 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:16.803053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:16.803085 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:53:16.803206 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:16.803213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:16.803292 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:16.804207 (+ 915us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:16.804309 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.805817 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.806930 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.807583 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.807683 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:16.810633 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:16.810640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:16.810640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:16.810657 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:16.814625 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:16.815170 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:16.815174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:16.815174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:16.815276 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:16.815516 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:16.815516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:16.815517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:16.815793 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:16.815982 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:16.816053 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:53:16.816212 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260806 06:53:16.817128 17830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Generated new TSK 0
I20260806 06:53:16.817291 17830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:53:16.845278 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:16.929342 17847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:16.929596 17847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:16.929637 17847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:16.930899 17847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:16.930968 17847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:16.931015 17847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:16.932520 17847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:16.932796 17847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:16.933053 17847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:16.938315 17852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:16.938330 17853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:16.938491 17847 server_base.cc:1034] running on GCE node
W20260806 06:53:16.938338 17855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:16.938794 17847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:16.939076 17847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:16.940222 17847 hybrid_clock.cc:648] HybridClock initialized: now 1785999196940188 us; error 50 us; skew 500 ppm
Aug 06 06:53:16 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999196, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:16.942978 17847 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:16.943434 17847 webserver.cc:460] Webserver started at http://127.3.247.129:36005/ using document root <none> and password file <none>
I20260806 06:53:16.943665 17847 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:16.943709 17847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:16.943861 17847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:16.944923 17847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "21ff454c71674d8898b5600a079d26bd"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "eb93ce5ae41e1e54c0cda259e806d8aa"
server_key_iv: "3ab0d9e5a3cb3b3896f3db832d2e08ef"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.945288 17847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "21ff454c71674d8898b5600a079d26bd"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "eb93ce5ae41e1e54c0cda259e806d8aa"
server_key_iv: "3ab0d9e5a3cb3b3896f3db832d2e08ef"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.946694 17847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:16.947471 17862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:16.947666 17847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:16.947767 17847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "21ff454c71674d8898b5600a079d26bd"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "eb93ce5ae41e1e54c0cda259e806d8aa"
server_key_iv: "3ab0d9e5a3cb3b3896f3db832d2e08ef"
server_key_version: "encryptionkey@0"
I20260806 06:53:16.947852 17847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:16.962484 17847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:16.964901 17847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:16.965080 17847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:16.965330 17847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:16.965655 17847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:16.965711 17847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:16.965757 17847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:16.965812 17847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:16.971962 17847 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:40353
I20260806 06:53:16.972054 17975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:40353 every 8 connection(s)
I20260806 06:53:16.972393 17847 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:53:16 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999196, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:16.980948 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17847
I20260806 06:53:16.981045 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:53:16.981278 4062 external_mini_cluster.cc:1428] Setting key c1b9e470ce34347eeae78873c22cf280
I20260806 06:53:16.983088 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:16.973208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:38661 (local address 127.3.247.190:38181)
0806 06:53:16.973303 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:16.973305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:16.973464 (+ 159us) server_negotiation.cc:407] Connection header received
0806 06:53:16.974045 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:16.974046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:16.974057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:16.974128 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:16.975488 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.976159 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.977176 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.977379 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.977453 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:16.979635 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:16.979636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:16.979637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:16.979644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:16.981356 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:16.981981 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:16.981982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:16.981983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:16.982062 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:16.982358 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:16.982359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:16.982359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:16.982492 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:16.982580 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:16.982888 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 06:53:16.982990 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 06:53:16.983304 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:16.973147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:38661)
0806 06:53:16.973314 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:16.973366 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:16.973952 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:16.974204 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:16.974208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:16.974800 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:16.975205 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:16.975214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.976249 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.976250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:16.977074 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:16.977077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:16.977465 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:16.977466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:16.977565 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:16.978132 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:16.978140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:16.979539 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:16.981466 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:16.981475 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:16.981477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:16.981877 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:16.982146 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:16.982147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:16.982147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:16.982279 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:16.982566 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:16.982573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:16.982735 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 06:53:16.982849 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:53:16.983026 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260806 06:53:16.983781 17976 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:16.983914 17976 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:16.984107 17976 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:16.984663 17785 ts_manager.cc:194] Registered new tserver with Master: 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:16.985307 17785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:38661
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:53:17.006683 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:17.089578 17981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.089819 17981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.089851 17981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.091037 17981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.091117 17981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.091163 17981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:17.092646 17981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.092849 17981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.093046 17981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.098080 17986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.098090 17987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.098116 17989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.098188 17981 server_base.cc:1034] running on GCE node
I20260806 06:53:17.098572 17981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.098830 17981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.100005 17981 hybrid_clock.cc:648] HybridClock initialized: now 1785999197099995 us; error 27 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.102721 17981 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:17.103266 17981 webserver.cc:460] Webserver started at http://127.3.247.130:42635/ using document root <none> and password file <none>
I20260806 06:53:17.103504 17981 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.103574 17981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.103751 17981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:17.104838 17981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "f1d5f9428f444930befe783c0508cb82"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "bc5429b06ed254b6310d7f382a344211"
server_key_iv: "e1047740d066666f395f6d078373239b"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.105216 17981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "f1d5f9428f444930befe783c0508cb82"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "bc5429b06ed254b6310d7f382a344211"
server_key_iv: "e1047740d066666f395f6d078373239b"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.106621 17981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:17.107399 17996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.107566 17981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:17.107667 17981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "f1d5f9428f444930befe783c0508cb82"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "bc5429b06ed254b6310d7f382a344211"
server_key_iv: "e1047740d066666f395f6d078373239b"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.107754 17981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:17.126597 17981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:17.129168 17981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:17.129346 17981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:17.129598 17981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:17.129988 17981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:17.130056 17981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.130103 17981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:17.130126 17981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.136041 17981 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44461
I20260806 06:53:17.136101 18109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44461 every 8 connection(s)
I20260806 06:53:17.136550 17981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:17.142006 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 17981
I20260806 06:53:17.142114 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:53:17.142350 4062 external_mini_cluster.cc:1428] Setting key 967e039a44f87e9c1b275512001e683b
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:17.146715 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.137347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:42155 (local address 127.3.247.190:38181)
0806 06:53:17.137409 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.137411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.137603 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:53:17.138207 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.138209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.138222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.138305 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:17.139315 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.139823 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.140786 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.140982 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.141040 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.143584 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:17.143585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:17.143586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:17.143595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:17.145132 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.145707 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.145708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.145708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.145754 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.146119 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.146120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.146120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.146224 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:17.146275 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.146593 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.146653 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:53:17.147054 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.137285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:42155)
0806 06:53:17.137463 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.137507 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.138068 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.138387 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.138391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.138999 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:17.139198 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.139202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.139910 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.139911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.140700 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.140702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.141059 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.141060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.141158 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.141635 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:17.141643 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:17.143494 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:17.145220 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.145228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.145230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.145636 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.145897 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.145898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.145898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.146052 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.146290 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:17.146296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:17.146462 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.146576 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.146757 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260806 06:53:17.147665 18110 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:17.147784 18110 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:17.148011 18110 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:17.148444 17785 ts_manager.cc:194] Registered new tserver with Master: f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
I20260806 06:53:17.149020 17785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:42155
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:53:17.169063 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:17.249872 18115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.250072 18115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.250094 18115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.251365 18115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.251442 18115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.251497 18115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:17.252983 18115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.253240 18115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.253494 18115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.258591 18120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.258616 18121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.258749 18115 server_base.cc:1034] running on GCE node
W20260806 06:53:17.258605 18123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.259054 18115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.259275 18115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.260442 18115 hybrid_clock.cc:648] HybridClock initialized: now 1785999197260422 us; error 36 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.262996 18115 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:17.263494 18115 webserver.cc:460] Webserver started at http://127.3.247.131:40011/ using document root <none> and password file <none>
I20260806 06:53:17.263748 18115 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.263796 18115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.263888 18115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:17.264881 18115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "98de812a7a504f1a9a51874829e7922c"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "84842bfb35c59e576dceb2389ec341d8"
server_key_iv: "ad8c2f38716091590374402605c6f787"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.265169 18115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "98de812a7a504f1a9a51874829e7922c"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "84842bfb35c59e576dceb2389ec341d8"
server_key_iv: "ad8c2f38716091590374402605c6f787"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.266485 18115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:53:17.267207 18130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.267370 18115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:17.267439 18115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "98de812a7a504f1a9a51874829e7922c"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "84842bfb35c59e576dceb2389ec341d8"
server_key_iv: "ad8c2f38716091590374402605c6f787"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.267491 18115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:17.297974 18115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:17.299904 18115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:17.300019 18115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:17.300206 18115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:17.300539 18115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:17.300572 18115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.300618 18115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:17.300645 18115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.307066 18115 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:44385
I20260806 06:53:17.307080 18243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:44385 every 8 connection(s)
I20260806 06:53:17.307446 18115 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:17.314230 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18115
I20260806 06:53:17.314318 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:53:17.314523 4062 external_mini_cluster.cc:1428] Setting key aeae01d11fefb47d47e49812b4e96bf2
I20260806 06:53:17.317468 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.308133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:50047 (local address 127.3.247.190:38181)
0806 06:53:17.308179 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.308182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.308401 (+ 219us) server_negotiation.cc:407] Connection header received
0806 06:53:17.308898 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.308899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.308910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.308959 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:17.309952 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.310486 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.311463 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.311686 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.311758 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.314103 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:17.314105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:17.314105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:17.314121 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:17.315806 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.316303 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.316304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.316305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.316358 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.316700 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.316700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.316701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.316805 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:17.316858 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.317308 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.317392 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:53:17.317694 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.308140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:50047)
0806 06:53:17.308295 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.308331 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.308809 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.309022 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.309025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.309598 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:17.309872 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.309878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.310564 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.310565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.311384 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.311387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.311773 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.311774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.311887 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.312388 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:17.312396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:17.313990 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:17.315912 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.315919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.315922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.316237 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.316452 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.316453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.316453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.316629 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.316904 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:17.316940 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:17.317125 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.317265 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.317414 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260806 06:53:17.318254 18244 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:17.318380 18244 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:17.318593 18244 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:17.318955 17785 ts_manager.cc:194] Registered new tserver with Master: 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385)
I20260806 06:53:17.319340 17785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:50047
I20260806 06:53:17.326985 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:17.334146 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.328352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.3.247.190:38181)
0806 06:53:17.328413 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.328416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.328540 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:53:17.328580 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.328580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.328590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.328642 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:17.329205 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.329680 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.330436 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.330615 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.330668 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.331781 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:17.331782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:17.331783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:17.331789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:17.333056 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.333368 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.333369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.333369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.333426 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.333596 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.333596 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.333596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.333766 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:17.333933 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.334018 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.334078 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:53:17.335752 17785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:53:17.337234 17785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:53:17.343634 18255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.339598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.0.0.1:58712)
0806 06:53:17.339691 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.339732 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.339794 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.340166 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.340168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.340170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.340242 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.340244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341248 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.341249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.342424 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.342426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343335 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.343423 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343439 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.343488 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.343537 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":49,"threads_started":1}
I20260806 06:53:17.343748 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.339601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.0.0.1:56216)
0806 06:53:17.339893 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.339905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.339962 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.340312 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.340313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.340315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.340364 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.340367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341690 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.341691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.342706 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.342708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343456 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.343523 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343527 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.343599 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.343645 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":10,"thread_start_us":93,"threads_started":1}
I20260806 06:53:17.343950 18259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.339792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44461 (local address 127.0.0.1:49372)
0806 06:53:17.339966 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.339976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.340017 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.340473 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.340474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.340476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.340509 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.340510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341733 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.341734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.342753 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.342755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343719 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.343799 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343804 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.343853 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.343888 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":142,"threads_started":1}
I20260806 06:53:17.344107 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.339720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.3.247.129:40353)
0806 06:53:17.339984 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.339990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.340001 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:53:17.340043 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.340045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.340062 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.340149 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.340348 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341138 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.342496 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343237 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343862 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343936 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.343962 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.344011 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":84,"threads_started":1}
I20260806 06:53:17.344133 18258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.339770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56216 (local address 127.3.247.131:44385)
0806 06:53:17.340089 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.340093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.340102 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:17.340137 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.340139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.340153 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.340221 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.340456 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341584 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.342800 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343352 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343429 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343492 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.343628 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.343673 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":76,"threads_started":1}
I20260806 06:53:17.344465 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.340071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49372 (local address 127.3.247.130:44461)
0806 06:53:17.340254 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.340259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.340268 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:17.340306 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.340308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.340323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.340399 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.340576 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.341647 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.342843 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.343573 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.343633 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.343684 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.344120 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.344379 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260806 06:53:17.344800 17910 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (DEFAULT_TABLE table=test-table [id=4b9d3558f73d4f4f9c901d085a865d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:17.344799 18178 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (DEFAULT_TABLE table=test-table [id=4b9d3558f73d4f4f9c901d085a865d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:17.345167 18178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:17.345172 17910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:17.345194 18044 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (DEFAULT_TABLE table=test-table [id=4b9d3558f73d4f4f9c901d085a865d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:17.345531 18044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:17.348093 18261 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Bootstrap starting.
I20260806 06:53:17.348483 18262 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Bootstrap starting.
I20260806 06:53:17.348825 18261 tablet_bootstrap.cc:654] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:17.349220 18262 tablet_bootstrap.cc:654] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:17.349488 18261 log.cc:824] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:17.349869 18262 log.cc:824] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:17.349921 18263 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Bootstrap starting.
I20260806 06:53:17.350395 18261 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: No bootstrap required, opened a new log
I20260806 06:53:17.350482 18261 ts_tablet_manager.cc:1397] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:53:17.350594 18263 tablet_bootstrap.cc:654] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:17.350610 18262 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: No bootstrap required, opened a new log
I20260806 06:53:17.350670 18262 ts_tablet_manager.cc:1397] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:53:17.351366 18263 log.cc:824] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:17.352152 18263 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: No bootstrap required, opened a new log
I20260806 06:53:17.352224 18263 ts_tablet_manager.cc:1397] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:17.352419 18261 raft_consensus.cc:348] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.352588 18261 raft_consensus.cc:374] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:17.352675 18261 raft_consensus.cc:729] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ff454c71674d8898b5600a079d26bd, State: Initialized, Role: FOLLOWER
I20260806 06:53:17.352798 18261 consensus_queue.cc:260] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.352834 18262 raft_consensus.cc:348] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.353021 18262 raft_consensus.cc:374] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:17.353065 18262 raft_consensus.cc:729] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98de812a7a504f1a9a51874829e7922c, State: Initialized, Role: FOLLOWER
I20260806 06:53:17.353087 18261 ts_tablet_manager.cc:1428] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:53:17.353195 18262 consensus_queue.cc:260] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.353237 17976 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:17.353497 18244 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:17.353487 18262 ts_tablet_manager.cc:1428] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:53:17.354508 18263 raft_consensus.cc:348] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.354694 18263 raft_consensus.cc:374] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:17.354739 18263 raft_consensus.cc:729] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d5f9428f444930befe783c0508cb82, State: Initialized, Role: FOLLOWER
I20260806 06:53:17.354844 18263 consensus_queue.cc:260] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.355121 18110 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:17.355322 18263 ts_tablet_manager.cc:1428] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 06:53:17.387116 18111 tablet.cc:2369] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:17.439621 18269 raft_consensus.cc:482] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:17.439751 18269 raft_consensus.cc:504] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.440172 18269 leader_election.cc:288] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:17.444073 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.440491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:51451 (local address 127.3.247.129:40353)
0806 06:53:17.440547 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.440549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.440578 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:53:17.440610 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.440611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.440621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.440676 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.440903 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.441682 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443105 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.443715 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443780 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.443788 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.443957 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.443999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:53:17.444072 18258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.440508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:36389 (local address 127.3.247.131:44385)
0806 06:53:17.440558 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.440560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.440914 (+ 354us) server_negotiation.cc:407] Connection header received
0806 06:53:17.440945 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.440947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.440957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.441007 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.441149 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.441897 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443105 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.443684 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443779 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.443788 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.443958 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.443999 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:53:17.444079 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.440431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.3.247.130:51451)
0806 06:53:17.440483 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.440496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.440543 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.440699 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.440700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.440703 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.440798 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.440800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.441760 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.441760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.443002 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.443004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.443788 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.443874 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.443876 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.443936 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.443986 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":3}
I20260806 06:53:17.444085 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.440432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.3.247.130:36389)
0806 06:53:17.440794 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.440805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.440868 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.441022 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.441023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.441024 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.441077 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.441079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.441970 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.441970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.443018 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.443020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.443788 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.443789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.443870 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.443874 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.443953 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.443986 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":156,"thread_start_us":126,"threads_started":1}
I20260806 06:53:17.444504 18198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0bdd75c33a4a7d9cd992f97ce1ef32" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98de812a7a504f1a9a51874829e7922c" is_pre_election: true
I20260806 06:53:17.444494 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0bdd75c33a4a7d9cd992f97ce1ef32" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ff454c71674d8898b5600a079d26bd" is_pre_election: true
I20260806 06:53:17.444666 18198 raft_consensus.cc:2454] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 0.
I20260806 06:53:17.444679 17930 raft_consensus.cc:2454] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 0.
I20260806 06:53:17.444967 17999 leader_election.cc:302] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd, f1d5f9428f444930befe783c0508cb82; no voters:
I20260806 06:53:17.445109 18269 raft_consensus.cc:2781] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:17.445164 18269 raft_consensus.cc:482] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:17.445194 18269 raft_consensus.cc:3037] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:17.445879 18269 raft_consensus.cc:504] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.446031 18269 leader_election.cc:288] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 election: Requested vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:17.446177 18198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0bdd75c33a4a7d9cd992f97ce1ef32" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98de812a7a504f1a9a51874829e7922c"
I20260806 06:53:17.446185 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0bdd75c33a4a7d9cd992f97ce1ef32" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ff454c71674d8898b5600a079d26bd"
I20260806 06:53:17.446260 18198 raft_consensus.cc:3037] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:17.446269 17930 raft_consensus.cc:3037] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:17.447154 17930 raft_consensus.cc:2454] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 1.
I20260806 06:53:17.447154 18198 raft_consensus.cc:2454] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 1.
I20260806 06:53:17.447337 17997 leader_election.cc:302] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82; no voters:
I20260806 06:53:17.447429 18269 raft_consensus.cc:2781] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:17.447551 18269 raft_consensus.cc:686] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 1 LEADER]: Becoming Leader. State: Replica: f1d5f9428f444930befe783c0508cb82, State: Running, Role: LEADER
I20260806 06:53:17.447629 18269 consensus_queue.cc:237] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.448484 17784 catalog_manager.cc:5495] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1d5f9428f444930befe783c0508cb82 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "f1d5f9428f444930befe783c0508cb82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } health_report { overall_health: HEALTHY } } }
I20260806 06:53:17.459475 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.456946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.3.247.130:44461)
0806 06:53:17.457013 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.457015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.457026 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:53:17.457065 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.457066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.457088 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.457168 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:17.457334 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.457884 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.458651 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.458868 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.458942 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.459116 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:17.459231 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:17.459326 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.459345 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.459366 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:53:17.461337 18198 raft_consensus.cc:1261] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Refusing update from remote peer f1d5f9428f444930befe783c0508cb82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:17.461337 17930 raft_consensus.cc:1261] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Refusing update from remote peer f1d5f9428f444930befe783c0508cb82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:17.461680 18271 consensus_queue.cc:1035] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:17.461856 18269 consensus_queue.cc:1035] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:17.464596 18275 mvcc.cc:205] Tried to move back new op lower bound from 7315452712798658560 to 7315452712745918464. Current Snapshot: MvccSnapshot[applied={T|T < 7315452712798658560}]
I20260806 06:53:17.465588 18276 mvcc.cc:205] Tried to move back new op lower bound from 7315452712798658560 to 7315452712745918464. Current Snapshot: MvccSnapshot[applied={T|T < 7315452712798658560 or (T in {7315452712798658560})}]
I20260806 06:53:17.466637 17785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:17.466741 17785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:17.467542 17785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Sending DeleteTablet for 3 replicas of tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32
I20260806 06:53:17.467890 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17847
I20260806 06:53:17.467924 18178 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:17 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56216
I20260806 06:53:17.467936 17910 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:17 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:58712
I20260806 06:53:17.467952 18044 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:17 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:49372
I20260806 06:53:17.468580 18285 tablet_replica.cc:330] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: stopping tablet replica
I20260806 06:53:17.468674 18285 raft_consensus.cc:2229] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:53:17.468760 18285 raft_consensus.cc:2258] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:17.469434 18285 ts_tablet_manager.cc:1905] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:17.469820 18284 tablet_replica.cc:330] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: stopping tablet replica
I20260806 06:53:17.469990 18284 raft_consensus.cc:2229] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:17.470145 18284 raft_consensus.cc:2258] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:17.470623 18284 ts_tablet_manager.cc:1905] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:17.471257 18285 ts_tablet_manager.cc:1918] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:17.471388 18285 log.cc:1196] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/0a0bdd75c33a4a7d9cd992f97ce1ef32
I20260806 06:53:17.471752 18285 ts_tablet_manager.cc:1939] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P f1d5f9428f444930befe783c0508cb82: Deleting consensus metadata
I20260806 06:53:17.472249 17771 catalog_manager.cc:4841] TS f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (table test-table [id=4b9d3558f73d4f4f9c901d085a865d85]) successfully deleted
I20260806 06:53:17.472697 18284 ts_tablet_manager.cc:1918] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:17.472781 18284 log.cc:1196] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/0a0bdd75c33a4a7d9cd992f97ce1ef32
I20260806 06:53:17.473069 18284 ts_tablet_manager.cc:1939] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 98de812a7a504f1a9a51874829e7922c: Deleting consensus metadata
I20260806 06:53:17.473493 17769 catalog_manager.cc:4841] TS 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (table test-table [id=4b9d3558f73d4f4f9c901d085a865d85]) successfully deleted
W20260806 06:53:17.473922 17771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:40353 (error 108)
I20260806 06:53:17.474136 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17981
I20260806 06:53:17.474411 18259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.474308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.0.0.1:58714)
0806 06:53:17.474352 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.474381 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:40353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:53:17.474511 17771 catalog_manager.cc:4584] TS 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32: Network error: Client connection negotiation failed: client connection to 127.3.247.129:40353: connect: Connection refused (error 111)
I20260806 06:53:17.480155 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18115
I20260806 06:53:17.484519 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 17753
I20260806 06:53:17.489674 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--webserver_interface=127.3.247.190
--webserver_port=45717
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:17.571287 18287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.571557 18287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.571586 18287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.572808 18287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:17.572862 18287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.572877 18287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.572890 18287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:17.572903 18287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:17.574502 18287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=45717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.574774 18287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.575030 18287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.584203 18293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.584195 18294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.584452 18287 server_base.cc:1034] running on GCE node
W20260806 06:53:17.584195 18296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.584825 18287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.585179 18287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.586352 18287 hybrid_clock.cc:648] HybridClock initialized: now 1785999197586347 us; error 36 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.588833 18287 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:17.589222 18287 webserver.cc:460] Webserver started at http://127.3.247.190:45717/ using document root <none> and password file <none>
I20260806 06:53:17.589427 18287 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.589480 18287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.590940 18287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:17.591583 18303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.591774 18287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:17.591862 18287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "a50d9607832b4152a0b5d6d492868a52"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "a8b644883b5f499edf04a7124d32f51d"
server_key_iv: "868ee0cdcb88987770d14c1011d4daa7"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.592255 18287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:17.606882 18287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:17.609226 18287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:17.609342 18287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:17.613246 18287 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38181
I20260806 06:53:17.613302 18355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38181 every 8 connection(s)
I20260806 06:53:17.613715 18287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:17.614910 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18287
I20260806 06:53:17.615214 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:40353
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=36005
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:17.615912 18356 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:53:17.616942 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap starting.
I20260806 06:53:17.625367 18356 log.cc:824] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:17.627614 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:17.627977 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap complete.
I20260806 06:53:17.630481 18356 raft_consensus.cc:348] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:17.630707 18356 raft_consensus.cc:729] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Initialized, Role: FOLLOWER
I20260806 06:53:17.630905 18356 consensus_queue.cc:260] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:17.630965 18356 raft_consensus.cc:388] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:17.631034 18356 raft_consensus.cc:482] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:17.631090 18356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:53:17.632110 18356 raft_consensus.cc:504] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:17.632196 18356 leader_election.cc:302] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a50d9607832b4152a0b5d6d492868a52; no voters:
I20260806 06:53:17.632422 18356 leader_election.cc:288] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:53:17.632458 18360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:53:17.632562 18360 raft_consensus.cc:686] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 LEADER]: Becoming Leader. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Running, Role: LEADER
I20260806 06:53:17.632709 18356 sys_catalog.cc:564] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:17.632723 18360 consensus_queue.cc:237] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:17.633077 18362 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a50d9607832b4152a0b5d6d492868a52. Latest consensus state: current_term: 2 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:17.633318 18362 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:17.633340 18361 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:17.633528 18361 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:17.633592 18366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:17.634558 18366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:17.634757 18366 catalog_manager.cc:1259] Loaded cluster ID: 074c096220294ece862d59fdb2c75878
I20260806 06:53:17.634956 18366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:53:17.635196 18366 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 06:53:17.639895 18366 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 06:53:17.642414 18366 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:17.642666 18366 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Loaded TSK: 0
I20260806 06:53:17.642920 18366 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:53:17.700841 18358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.701074 18358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.701115 18358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.702425 18358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.702490 18358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.702544 18358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:17.703999 18358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:40353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=36005
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.704257 18358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.704495 18358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.709547 18383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.709558 18386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.709560 18384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.710040 18358 server_base.cc:1034] running on GCE node
I20260806 06:53:17.710248 18358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.710487 18358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.711655 18358 hybrid_clock.cc:648] HybridClock initialized: now 1785999197711643 us; error 30 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.714372 18358 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:17.714915 18358 webserver.cc:460] Webserver started at http://127.3.247.129:36005/ using document root <none> and password file <none>
I20260806 06:53:17.715178 18358 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.715287 18358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.717121 18358 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:53:17.717967 18393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.718293 18358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:17.718394 18358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "21ff454c71674d8898b5600a079d26bd"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "eb93ce5ae41e1e54c0cda259e806d8aa"
server_key_iv: "3ab0d9e5a3cb3b3896f3db832d2e08ef"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.718853 18358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:17.737288 18358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:17.739888 18358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:17.740053 18358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:17.740337 18358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:17.740826 18400 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:53:17.742353 18358 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:53:17.742414 18358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:53:17.742448 18358 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:53:17.743305 18358 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:53:17.743353 18358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:17.743423 18400 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Bootstrap starting.
I20260806 06:53:17.750874 18358 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:40353
I20260806 06:53:17.751142 18507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:40353 every 8 connection(s)
I20260806 06:53:17.751358 18358 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:17.753464 18400 log.cc:824] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:17.756660 18400 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:17.756996 18400 tablet_bootstrap.cc:492] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Bootstrap complete.
I20260806 06:53:17.758002 18400 ts_tablet_manager.cc:1397] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260806 06:53:17.759382 18400 raft_consensus.cc:348] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.759636 18400 raft_consensus.cc:729] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ff454c71674d8898b5600a079d26bd, State: Initialized, Role: FOLLOWER
I20260806 06:53:17.759799 18400 consensus_queue.cc:260] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } }
I20260806 06:53:17.760123 18400 ts_tablet_manager.cc:1428] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 06:53:17.760646 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18358
I20260806 06:53:17.760960 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:44461
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=42635
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:17.769158 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.752102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:43553)
0806 06:53:17.753684 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.753734 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.754480 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.758442 (+ 3962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.758446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.758810 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:17.759088 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.759095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.760819 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.760820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.762144 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.762148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.762957 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.762957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.763054 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.763620 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:17.763628 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:17.765228 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:17.767149 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.767156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.767158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.767453 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.767785 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.767786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.767786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.767923 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.768411 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:17.768418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:17.768585 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.768707 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.768902 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1526,"thread_start_us":92,"threads_started":1}
I20260806 06:53:17.769240 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.752214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:43553 (local address 127.3.247.190:38181)
0806 06:53:17.757951 (+ 5737us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.757962 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.758008 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:53:17.758146 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.758154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.758212 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.758380 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:17.759238 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.760677 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.762260 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.762857 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.762965 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.765357 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:17.765361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:17.765361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:17.765372 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:17.767070 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.767577 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.767580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.767581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.767707 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.768005 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.768006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.768006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.768331 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:17.768434 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.768799 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.768963 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5667,"thread_start_us":87,"threads_started":1}
I20260806 06:53:17.770141 18508 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:17.770244 18508 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:17.770445 18508 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:17.771279 18320 ts_manager.cc:194] Registered new tserver with Master: 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:17.772603 18320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:43553
I20260806 06:53:17.773634 18508 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:17.778858 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.773077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.0.0.1:58722)
0806 06:53:17.774180 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.774220 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.774284 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.774638 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.774642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.774644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:17.774714 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.774716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.776060 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.776062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.777512 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.777515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.778560 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.778561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.778646 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.778658 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.778722 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.778774 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":98,"threads_started":1}
I20260806 06:53:17.779263 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.773333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.3.247.129:40353)
0806 06:53:17.774423 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.774428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.774437 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:17.774474 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.774476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.774494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.774566 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:17.774797 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.775944 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.777611 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.778403 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.778472 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.778526 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.779137 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.779183 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":74,"threads_started":1}
I20260806 06:53:17.779639 18442 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:17 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:58722
I20260806 06:53:17.780318 18517 tablet_replica.cc:330] stopping tablet replica
I20260806 06:53:17.780444 18517 raft_consensus.cc:2229] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:17.780521 18517 raft_consensus.cc:2258] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:17.780820 18517 ts_tablet_manager.cc:1905] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:17.782958 18517 ts_tablet_manager.cc:1918] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:17.783054 18517 log.cc:1196] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/0a0bdd75c33a4a7d9cd992f97ce1ef32
I20260806 06:53:17.783298 18517 ts_tablet_manager.cc:1939] T 0a0bdd75c33a4a7d9cd992f97ce1ef32 P 21ff454c71674d8898b5600a079d26bd: Deleting consensus metadata
I20260806 06:53:17.783859 18306 catalog_manager.cc:4841] TS 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353): tablet 0a0bdd75c33a4a7d9cd992f97ce1ef32 (table test-table [id=4b9d3558f73d4f4f9c901d085a865d85]) successfully deleted
W20260806 06:53:17.851226 18514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.851444 18514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.851468 18514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.852674 18514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.852718 18514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.852746 18514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:17.854231 18514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:44461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=42635
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.854478 18514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.854717 18514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.859699 18525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.859709 18522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.859709 18523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.859860 18514 server_base.cc:1034] running on GCE node
I20260806 06:53:17.860335 18514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.860534 18514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.861677 18514 hybrid_clock.cc:648] HybridClock initialized: now 1785999197861639 us; error 53 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.864259 18514 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:17.864698 18514 webserver.cc:460] Webserver started at http://127.3.247.130:42635/ using document root <none> and password file <none>
I20260806 06:53:17.864925 18514 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.864982 18514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.866420 18514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:17.867058 18532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.867249 18514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:17.867349 18514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "f1d5f9428f444930befe783c0508cb82"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "bc5429b06ed254b6310d7f382a344211"
server_key_iv: "e1047740d066666f395f6d078373239b"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.867691 18514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:17.879434 18514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:17.882091 18514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:17.882261 18514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:17.882498 18514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:17.882798 18514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:17.882853 18514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.882893 18514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:17.882921 18514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.888765 18514 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44461
I20260806 06:53:17.888820 18645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44461 every 8 connection(s)
I20260806 06:53:17.889178 18514 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:17.890501 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18514
I20260806 06:53:17.890790 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:44385
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=40011
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:17.899147 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.890011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:59709 (local address 127.3.247.190:38181)
0806 06:53:17.890058 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:17.890060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:17.890276 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:53:17.890845 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:17.890847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:17.890865 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:17.890915 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:17.891856 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.892372 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.893371 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.893608 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.893693 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.896184 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:17.896186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:17.896186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:17.896194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:17.897629 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.898114 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.898116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.898116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.898181 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:17.898454 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:17.898455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:17.898455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:17.898581 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:17.898662 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:17.898949 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:53:17.899051 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:53:17.899333 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:17.889992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:59709)
0806 06:53:17.890142 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:17.890185 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:17.890715 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:17.890992 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:17.890996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:17.891558 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:17.891748 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.891753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.892468 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.892470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.893267 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:17.893269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:17.893705 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:17.893706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:17.893887 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:17.894605 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:17.894619 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:17.896079 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:17.897732 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.897740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.897742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.898018 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.898250 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:17.898251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:17.898252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:17.898368 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:17.898653 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:17.898660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:17.898809 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 06:53:17.898923 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:53:17.899083 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260806 06:53:17.899837 18646 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:17.899950 18646 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:17.900138 18646 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:17.900578 18320 ts_manager.cc:194] Registered new tserver with Master: f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
I20260806 06:53:17.900974 18320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:59709
W20260806 06:53:17.976016 18649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:17.976248 18649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:17.976289 18649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:17.977602 18649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:17.977669 18649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:17.977723 18649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:17.979292 18649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:44385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=40011
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:17.979578 18649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:17.979839 18649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:17.984907 18655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.984912 18654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:17.984915 18657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:17.985245 18649 server_base.cc:1034] running on GCE node
I20260806 06:53:17.985440 18649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:17.985642 18649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:17.986780 18649 hybrid_clock.cc:648] HybridClock initialized: now 1785999197986763 us; error 32 us; skew 500 ppm
Aug 06 06:53:17 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:17.989210 18649 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:17.989593 18649 webserver.cc:460] Webserver started at http://127.3.247.131:40011/ using document root <none> and password file <none>
I20260806 06:53:17.989912 18649 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:17.990002 18649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:17.991364 18649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:17.991976 18664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:17.992174 18649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:53:17.992271 18649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "98de812a7a504f1a9a51874829e7922c"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "84842bfb35c59e576dceb2389ec341d8"
server_key_iv: "ad8c2f38716091590374402605c6f787"
server_key_version: "encryptionkey@0"
I20260806 06:53:17.992622 18649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:18.003608 18649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:18.006168 18649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:18.006343 18649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:18.006634 18649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:18.007020 18649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:18.007066 18649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.007095 18649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:18.007114 18649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.013667 18649 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:44385
I20260806 06:53:18.013826 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:44385 every 8 connection(s)
I20260806 06:53:18.014113 18649 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:53:18.016083 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18649
Aug 06 06:53:18 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999197, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:18.023440 18782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.016500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.3.247.190:38181)
0806 06:53:18.016617 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.016619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.016636 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:53:18.016725 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.016726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.016739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.016806 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:18.017350 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.017956 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.018713 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.018936 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.019020 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.020317 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:18.020318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:18.020319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:18.020325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:18.022116 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:18.022569 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:18.022570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:18.022571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:18.022653 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:18.022852 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:18.022852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:18.022853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:18.022969 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:18.023163 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.023201 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.023287 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260806 06:53:18.024091 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.014811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:38391 (local address 127.3.247.190:38181)
0806 06:53:18.014882 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.014884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.015127 (+ 243us) server_negotiation.cc:407] Connection header received
0806 06:53:18.015674 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.015676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.015689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.015740 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:18.016938 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.017434 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.018534 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.018748 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.018824 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.020994 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:18.020995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:18.020996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:18.021003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:18.022589 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:18.023037 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:18.023038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:18.023039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:18.023098 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:18.023347 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:18.023348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:18.023348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:18.023503 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:18.023569 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.023922 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.024008 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:53:18.024363 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.014833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:38391)
0806 06:53:18.015010 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.015045 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.015556 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.015816 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.015819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.016534 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:18.016845 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.016851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.017528 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.017529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.018446 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.018448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.018830 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.018831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.018964 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.019503 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:18.019514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:18.020893 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:18.022712 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:18.022718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:18.022721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:18.022960 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:18.023165 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:18.023166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:18.023166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:18.023280 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:18.023566 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:18.023572 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:18.023777 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.023887 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.024081 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260806 06:53:18.024784 18778 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38181
I20260806 06:53:18.024880 18778 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:18.025029 18778 heartbeater.cc:511] Master 127.3.247.190:38181 requested a full tablet report, sending...
I20260806 06:53:18.025406 18320 ts_manager.cc:194] Registered new tserver with Master: 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385)
I20260806 06:53:18.025863 18320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:38391
I20260806 06:53:18.035279 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:18.038566 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.035683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.3.247.190:38181)
0806 06:53:18.035762 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.035764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.035888 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:53:18.036005 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.036006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.036048 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.036130 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:18.036289 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.036777 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.037681 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.037952 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.038024 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.038210 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:18.038333 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:18.038432 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.038457 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.038478 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:53:18.038938 18320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:53:18.040549 18320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:53:18.042835 18442 tablet_service.cc:1467] Processing CreateTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 (DEFAULT_TABLE table=test-table [id=2c2411e675d74bc8833a388695747ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:18.043046 18442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d547d7dae944a5a8bd2fffc4af27a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:18.044351 18400 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Bootstrap starting.
I20260806 06:53:18.044687 18400 tablet_bootstrap.cc:654] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:18.045307 18400 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: No bootstrap required, opened a new log
I20260806 06:53:18.045359 18400 ts_tablet_manager.cc:1397] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:18.045459 18400 raft_consensus.cc:348] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.045527 18400 raft_consensus.cc:374] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:18.045576 18400 raft_consensus.cc:729] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ff454c71674d8898b5600a079d26bd, State: Initialized, Role: FOLLOWER
I20260806 06:53:18.045615 18400 consensus_queue.cc:260] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.045755 18400 ts_tablet_manager.cc:1428] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.045861 18513 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:18.045918 18513 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.046206 18513 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
I20260806 06:53:18.047019 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.042376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.0.0.1:56218)
0806 06:53:18.042441 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.042455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.042497 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.043030 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.043031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.043032 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.043076 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.043077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.044470 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.044471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.045504 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.045515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.046752 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.046753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.046825 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.046828 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.046858 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.046909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:53:18.047432 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.042468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.3.247.131:44385)
0806 06:53:18.042791 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.042795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.042803 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:18.042834 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.042836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.042848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.043177 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.043255 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.044346 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.045627 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.046666 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.047204 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.047282 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.047306 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.047353 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":96,"threads_started":1}
I20260806 06:53:18.048151 18712 tablet_service.cc:1467] Processing CreateTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 (DEFAULT_TABLE table=test-table [id=2c2411e675d74bc8833a388695747ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:18.048344 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.042558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.3.247.130:44461)
0806 06:53:18.042762 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.042767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.042777 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:53:18.042817 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.042819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.042841 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.042925 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.043204 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.044346 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.047205 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.047978 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.048057 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.048110 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.048221 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.048262 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
I20260806 06:53:18.048513 18712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d547d7dae944a5a8bd2fffc4af27a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:18.048580 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.042432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44461 (local address 127.0.0.1:49390)
0806 06:53:18.042592 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.042601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.042640 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.043010 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.043011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.043013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.043065 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.043066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.044946 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.044947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.047045 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.047047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.048058 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.048059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.048121 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.048126 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.048458 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.048505 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260806 06:53:18.049360 18580 tablet_service.cc:1467] Processing CreateTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 (DEFAULT_TABLE table=test-table [id=2c2411e675d74bc8833a388695747ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:18.049690 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d547d7dae944a5a8bd2fffc4af27a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:18.050169 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.046328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.3.247.129:49271)
0806 06:53:18.046375 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.046384 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.046429 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.046720 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.046721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.046724 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.047076 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.047079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.048060 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.048061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.049139 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.049142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.049932 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.049933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.050017 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.050021 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.050053 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.050094 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:53:18.050515 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.046373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49271 (local address 127.3.247.131:44385)
0806 06:53:18.046584 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.046585 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.046593 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:18.046613 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.046613 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.046622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.047243 (+ 621us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.047285 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.047961 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.049242 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.049826 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.050323 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.050335 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.050353 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.050391 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
W20260806 06:53:18.051000 18394 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:18.051405 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.046415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44461 (local address 127.3.247.129:39973)
0806 06:53:18.047187 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.047197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.047242 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.047405 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.047406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.047407 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.047468 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.047470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.049199 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.049200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.050341 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.050343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.051152 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.051153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.051220 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.051224 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.051282 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.051319 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":64,"threads_started":1}
I20260806 06:53:18.051465 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.046479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:39973 (local address 127.3.247.130:44461)
0806 06:53:18.046842 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.046844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.047291 (+ 447us) server_negotiation.cc:407] Connection header received
0806 06:53:18.047311 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.047311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.047320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.047373 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.047750 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.049110 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.050450 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.051031 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.051111 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.051119 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.051312 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.051356 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":101,"threads_started":1}
W20260806 06:53:18.051983 18396 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:18.052034 18396 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:18.052191 18788 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 06:53:18.052178 18792 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Bootstrap starting.
I20260806 06:53:18.053110 18792 tablet_bootstrap.cc:654] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:18.053125 18793 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Bootstrap starting.
I20260806 06:53:18.053424 18792 log.cc:824] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:18.053952 18793 tablet_bootstrap.cc:654] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:18.054236 18792 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: No bootstrap required, opened a new log
I20260806 06:53:18.054224 18793 log.cc:824] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:18.054322 18792 ts_tablet_manager.cc:1397] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:18.054934 18793 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: No bootstrap required, opened a new log
I20260806 06:53:18.054983 18793 ts_tablet_manager.cc:1397] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:53:18.055856 18792 raft_consensus.cc:348] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.056038 18792 raft_consensus.cc:374] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:18.056085 18792 raft_consensus.cc:729] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98de812a7a504f1a9a51874829e7922c, State: Initialized, Role: FOLLOWER
I20260806 06:53:18.056219 18792 consensus_queue.cc:260] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.056474 18778 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:18.056484 18792 ts_tablet_manager.cc:1428] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:18.056773 18793 raft_consensus.cc:348] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.056913 18793 raft_consensus.cc:374] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:18.056939 18793 raft_consensus.cc:729] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d5f9428f444930befe783c0508cb82, State: Initialized, Role: FOLLOWER
I20260806 06:53:18.057065 18793 consensus_queue.cc:260] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.057327 18793 ts_tablet_manager.cc:1428] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:53:18.057338 18646 heartbeater.cc:503] Master 127.3.247.190:38181 was elected leader, sending a full tablet report...
I20260806 06:53:18.110752 18797 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:18.110903 18797 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.111224 18797 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:18.114069 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.111487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:44987 (local address 127.3.247.131:44385)
0806 06:53:18.111535 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.111538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.111550 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:53:18.111601 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.111602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.111612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.111663 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.111822 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.112311 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113264 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.113665 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113752 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.113762 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.113959 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.114000 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:53:18.114073 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.111397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.3.247.130:44987)
0806 06:53:18.111469 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.111482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.111543 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.111668 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.111669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.111670 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.111747 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.111749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.112398 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.112399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.113167 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.113169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.113749 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.113862 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.113865 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.113948 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.113992 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":1}
I20260806 06:53:18.114226 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.111554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:45393 (local address 127.3.247.129:40353)
0806 06:53:18.111598 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.111600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.111715 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:53:18.111776 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.111777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.111786 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.111873 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:18.112034 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.112492 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113424 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.113848 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113935 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.113942 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.114093 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.114133 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:53:18.114370 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.111399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.3.247.130:45393)
0806 06:53:18.111642 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.111656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.111719 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.111885 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:18.111885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:18.111887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:18.111958 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.111959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.112580 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.112581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.113321 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:18.113323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.113934 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.113935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:18.114016 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.114019 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:18.114087 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:53:18.114270 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":85,"thread_start_us":111,"threads_started":1}
I20260806 06:53:18.114360 18732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d547d7dae944a5a8bd2fffc4af27a2" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98de812a7a504f1a9a51874829e7922c" is_pre_election: true
I20260806 06:53:18.114547 18732 raft_consensus.cc:2454] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 0.
I20260806 06:53:18.114678 18462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d547d7dae944a5a8bd2fffc4af27a2" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ff454c71674d8898b5600a079d26bd" is_pre_election: true
I20260806 06:53:18.114807 18462 raft_consensus.cc:2454] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 0.
I20260806 06:53:18.114794 18533 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82; no voters:
I20260806 06:53:18.114944 18797 raft_consensus.cc:2781] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:18.115018 18797 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:18.115043 18797 raft_consensus.cc:3037] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:18.115721 18797 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.115875 18797 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 election: Requested vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353)
I20260806 06:53:18.116024 18732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d547d7dae944a5a8bd2fffc4af27a2" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98de812a7a504f1a9a51874829e7922c"
I20260806 06:53:18.116055 18462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d547d7dae944a5a8bd2fffc4af27a2" candidate_uuid: "f1d5f9428f444930befe783c0508cb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ff454c71674d8898b5600a079d26bd"
I20260806 06:53:18.116124 18462 raft_consensus.cc:3037] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:18.116111 18732 raft_consensus.cc:3037] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:18.117019 18732 raft_consensus.cc:2454] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 1.
I20260806 06:53:18.117019 18462 raft_consensus.cc:2454] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5f9428f444930befe783c0508cb82 in term 1.
I20260806 06:53:18.117228 18535 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd, f1d5f9428f444930befe783c0508cb82; no voters:
I20260806 06:53:18.117359 18797 raft_consensus.cc:2781] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:18.117549 18797 raft_consensus.cc:686] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 1 LEADER]: Becoming Leader. State: Replica: f1d5f9428f444930befe783c0508cb82, State: Running, Role: LEADER
I20260806 06:53:18.117661 18797 consensus_queue.cc:237] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.118631 18320 catalog_manager.cc:5495] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1d5f9428f444930befe783c0508cb82 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "f1d5f9428f444930befe783c0508cb82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } health_report { overall_health: UNKNOWN } } }
I20260806 06:53:18.125032 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.122546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.3.247.130:44461)
0806 06:53:18.122632 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:18.122634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:18.122663 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:53:18.122743 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:18.122744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:18.122762 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:18.122832 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:18.122984 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.123563 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.124328 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:18.124515 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:18.124565 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:18.124725 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:18.124821 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:18.124905 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:18.124927 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 06:53:18.124947 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:53:18.127013 18732 raft_consensus.cc:1261] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Refusing update from remote peer f1d5f9428f444930befe783c0508cb82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:18.127027 18462 raft_consensus.cc:1261] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Refusing update from remote peer f1d5f9428f444930befe783c0508cb82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:18.127357 18799 consensus_queue.cc:1035] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:18.127558 18797 consensus_queue.cc:1035] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:18.129734 18801 mvcc.cc:205] Tried to move back new op lower bound from 7315452715525181440 to 7315452715490652160. Current Snapshot: MvccSnapshot[applied={T|T < 7315452715525181440}]
I20260806 06:53:18.130220 18802 mvcc.cc:205] Tried to move back new op lower bound from 7315452715525181440 to 7315452715490652160. Current Snapshot: MvccSnapshot[applied={T|T < 7315452715525181440}]
I20260806 06:53:18.130954 18803 mvcc.cc:205] Tried to move back new op lower bound from 7315452715525181440 to 7315452715490652160. Current Snapshot: MvccSnapshot[applied={T|T < 7315452715525181440 or (T in {7315452715525181440})}]
I20260806 06:53:18.132593 18319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:18.132687 18319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:18.133548 18319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Sending DeleteTablet for 3 replicas of tablet a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:18.133852 18712 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:18 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:56218
I20260806 06:53:18.133889 18580 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:18 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:49390
I20260806 06:53:18.134056 18442 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d547d7dae944a5a8bd2fffc4af27a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:18 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:58722
I20260806 06:53:18.134172 18517 tablet_replica.cc:330] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: stopping tablet replica
I20260806 06:53:18.134235 18517 raft_consensus.cc:2229] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:18.134289 18517 raft_consensus.cc:2258] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:18.134357 18811 tablet_replica.cc:330] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: stopping tablet replica
I20260806 06:53:18.134399 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18358
I20260806 06:53:18.134447 18811 raft_consensus.cc:2229] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:53:18.134557 18811 raft_consensus.cc:2258] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:18.134670 18812 tablet_replica.cc:330] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: stopping tablet replica
I20260806 06:53:18.134757 18812 raft_consensus.cc:2229] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:18.134833 18812 raft_consensus.cc:2258] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:18.135286 18812 ts_tablet_manager.cc:1905] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:18.135432 18811 ts_tablet_manager.cc:1905] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:18.137054 18811 ts_tablet_manager.cc:1918] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:18.137051 18812 ts_tablet_manager.cc:1918] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:18.137117 18811 log.cc:1196] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:18.137117 18812 log.cc:1196] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:18.137408 18811 ts_tablet_manager.cc:1939] T a8d547d7dae944a5a8bd2fffc4af27a2 P f1d5f9428f444930befe783c0508cb82: Deleting consensus metadata
I20260806 06:53:18.137408 18812 ts_tablet_manager.cc:1939] T a8d547d7dae944a5a8bd2fffc4af27a2 P 98de812a7a504f1a9a51874829e7922c: Deleting consensus metadata
I20260806 06:53:18.137805 18306 catalog_manager.cc:4841] TS f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): tablet a8d547d7dae944a5a8bd2fffc4af27a2 (table test-table [id=2c2411e675d74bc8833a388695747ed8]) successfully deleted
I20260806 06:53:18.137909 18304 catalog_manager.cc:4841] TS 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): tablet a8d547d7dae944a5a8bd2fffc4af27a2 (table test-table [id=2c2411e675d74bc8833a388695747ed8]) successfully deleted
W20260806 06:53:18.142510 18306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:40353 (error 108)
I20260806 06:53:18.142771 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18514
I20260806 06:53:18.143646 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.143341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:40353 (local address 127.0.0.1:58730)
0806 06:53:18.143567 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.143605 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:40353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
W20260806 06:53:18.143798 18306 catalog_manager.cc:4584] TS 21ff454c71674d8898b5600a079d26bd (127.3.247.129:40353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8d547d7dae944a5a8bd2fffc4af27a2: Network error: Client connection negotiation failed: client connection to 127.3.247.129:40353: connect: Connection refused (error 111)
I20260806 06:53:18.147887 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18649
I20260806 06:53:18.152338 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18287
I20260806 06:53:18.157727 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--webserver_interface=127.3.247.190
--webserver_port=45717
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:18.239799 18814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:18.240020 18814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:18.240043 18814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:18.241169 18814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:18.241206 18814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:18.241230 18814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:18.241242 18814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:18.241254 18814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:18.243245 18814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=45717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:18.243446 18814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:18.243615 18814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:18.248588 18820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.248594 18822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.248594 18819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.248893 18814 server_base.cc:1034] running on GCE node
I20260806 06:53:18.249150 18814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:18.249468 18814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:18.250644 18814 hybrid_clock.cc:648] HybridClock initialized: now 1785999198250632 us; error 29 us; skew 500 ppm
Aug 06 06:53:18 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:18.253254 18814 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:18.253759 18814 webserver.cc:460] Webserver started at http://127.3.247.190:45717/ using document root <none> and password file <none>
I20260806 06:53:18.254050 18814 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:18.254097 18814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:18.255507 18814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:18.256189 18829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.256371 18814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:53:18.256464 18814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "a50d9607832b4152a0b5d6d492868a52"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "a8b644883b5f499edf04a7124d32f51d"
server_key_iv: "868ee0cdcb88987770d14c1011d4daa7"
server_key_version: "encryptionkey@0"
I20260806 06:53:18.256819 18814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:18.278375 18814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:18.281255 18814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:18.281420 18814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:18.285818 18814 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38181
I20260806 06:53:18.285872 18881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38181 every 8 connection(s)
I20260806 06:53:18.286296 18814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:18.287839 18882 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:53:18.288522 18882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap starting.
I20260806 06:53:18.292800 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18814
I20260806 06:53:18.293152 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:40353
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=36005
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:18.297439 18882 log.cc:824] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:18.299852 18882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:18.300185 18882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52: Bootstrap complete.
I20260806 06:53:18.302531 18882 raft_consensus.cc:348] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:18.302754 18882 raft_consensus.cc:729] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Initialized, Role: FOLLOWER
I20260806 06:53:18.302947 18882 consensus_queue.cc:260] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:18.303012 18882 raft_consensus.cc:388] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:18.303062 18882 raft_consensus.cc:482] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:18.303112 18882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:53:18.304143 18882 raft_consensus.cc:504] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:18.304255 18882 leader_election.cc:302] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a50d9607832b4152a0b5d6d492868a52; no voters:
I20260806 06:53:18.304452 18882 leader_election.cc:288] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:53:18.304494 18886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:53:18.304615 18886 raft_consensus.cc:686] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [term 3 LEADER]: Becoming Leader. State: Replica: a50d9607832b4152a0b5d6d492868a52, State: Running, Role: LEADER
I20260806 06:53:18.304710 18886 consensus_queue.cc:237] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } }
I20260806 06:53:18.304750 18882 sys_catalog.cc:564] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:18.304996 18887 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:18.305083 18887 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:18.305315 18888 sys_catalog.cc:455] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a50d9607832b4152a0b5d6d492868a52. Latest consensus state: current_term: 3 leader_uuid: "a50d9607832b4152a0b5d6d492868a52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a50d9607832b4152a0b5d6d492868a52" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38181 } } }
I20260806 06:53:18.305418 18888 sys_catalog.cc:458] T 00000000000000000000000000000000 P a50d9607832b4152a0b5d6d492868a52 [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:18.305557 18891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:18.306442 18891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:18.306558 18891 catalog_manager.cc:1259] Loaded cluster ID: 074c096220294ece862d59fdb2c75878
I20260806 06:53:18.306859 18891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 06:53:18.307089 18891 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f57c6aa19ad google::LogMessage::Fail() at ??:0
@ 0x7f57c6aa488a google::LogMessage::SendToLog() at ??:0
@ 0x7f57c6aa14de google::LogMessage::Flush() at ??:0
@ 0x7f57c6aa2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f57ce9c20f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f57ce9cb8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f57c896f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f57c8964954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f57c97c86db start_thread at ??:0
@ 0x7f57c559f61f clone at ??:0
W20260806 06:53:18.392127 18884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:18.392342 18884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:18.392367 18884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:18.393646 18884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:18.393694 18884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:18.393723 18884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:18.395305 18884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:40353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=36005
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:18.395546 18884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:18.395818 18884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:18.401623 18919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.401629 18918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.401876 18884 server_base.cc:1034] running on GCE node
W20260806 06:53:18.401976 18921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.402189 18884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:18.402486 18884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:18.403669 18884 hybrid_clock.cc:648] HybridClock initialized: now 1785999198403656 us; error 26 us; skew 500 ppm
Aug 06 06:53:18 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:18.406555 18884 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:18.407030 18884 webserver.cc:460] Webserver started at http://127.3.247.129:36005/ using document root <none> and password file <none>
I20260806 06:53:18.407254 18884 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:18.407326 18884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:18.408937 18884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:18.409677 18928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.409889 18884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:18.409957 18884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "21ff454c71674d8898b5600a079d26bd"
format_stamp: "Formatted at 2026-08-06 06:53:16 on dist-test-slave-0kls"
server_key: "eb93ce5ae41e1e54c0cda259e806d8aa"
server_key_iv: "3ab0d9e5a3cb3b3896f3db832d2e08ef"
server_key_version: "encryptionkey@0"
I20260806 06:53:18.410296 18884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:18.426669 18884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:18.429114 18884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:18.429262 18884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:18.429533 18884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:18.430078 18935 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:53:18.432085 18884 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:53:18.432158 18884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:53:18.432205 18884 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:53:18.432843 18884 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:53:18.432899 18884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:18.432978 18935 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Bootstrap starting.
I20260806 06:53:18.440132 18884 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:40353
I20260806 06:53:18.440527 18884 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:18.444177 18935 log.cc:824] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:18.444612 19042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:40353 every 8 connection(s)
I20260806 06:53:18.447259 18935 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:18.447602 18935 tablet_bootstrap.cc:492] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Bootstrap complete.
I20260806 06:53:18.448824 18935 ts_tablet_manager.cc:1397] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260806 06:53:18.449010 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 18884
I20260806 06:53:18.449278 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:44461
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=42635
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:18.450479 18935 raft_consensus.cc:348] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.450748 18935 raft_consensus.cc:729] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ff454c71674d8898b5600a079d26bd, State: Initialized, Role: FOLLOWER
I20260806 06:53:18.450918 18935 consensus_queue.cc:260] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:18.451267 18935 ts_tablet_manager.cc:1428] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260806 06:53:18.518563 19046 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:53:18.445270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:39809)
0806 06:53:18.445453 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.445498 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:18.446059 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:18.518341 (+ 72282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
W20260806 06:53:18.518900 19043 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:38181 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:38181: Client connection negotiation failed: client connection to 127.3.247.190:38181: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:53:18.519291 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.519151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:38023)
0806 06:53:18.519223 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.519268 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:53:18.519680 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.519593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:37591)
0806 06:53:18.519640 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.519663 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:53:18.519811 19043 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:53:18.539820 19047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:18.540051 19047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:18.540091 19047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:18.541239 19047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:18.541311 19047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:18.541356 19047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:18.542884 19047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:44461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=42635
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:18.543150 19047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:18.543385 19047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:18.548296 19053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.548346 19054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.548386 19047 server_base.cc:1034] running on GCE node
W20260806 06:53:18.548352 19056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.548707 19047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:18.548964 19047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:18.550136 19047 hybrid_clock.cc:648] HybridClock initialized: now 1785999198550111 us; error 34 us; skew 500 ppm
Aug 06 06:53:18 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:18.552608 19047 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:18.552994 19047 webserver.cc:460] Webserver started at http://127.3.247.130:42635/ using document root <none> and password file <none>
I20260806 06:53:18.553184 19047 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:18.553225 19047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:18.554607 19047 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:18.555245 19063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.555434 19047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:18.555526 19047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "f1d5f9428f444930befe783c0508cb82"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "bc5429b06ed254b6310d7f382a344211"
server_key_iv: "e1047740d066666f395f6d078373239b"
server_key_version: "encryptionkey@0"
I20260806 06:53:18.555846 19047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:18.575841 19047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:18.578456 19047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:18.578662 19047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:18.578920 19047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:18.579267 19047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:18.579324 19047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.579360 19047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:18.579387 19047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.585386 19047 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44461
I20260806 06:53:18.585424 19176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44461 every 8 connection(s)
I20260806 06:53:18.585872 19047 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:18.586975 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.586581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:32975)
0806 06:53:18.586728 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.586802 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260806 06:53:18.587401 19177 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:38181 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:38181: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
I20260806 06:53:18.587685 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.587592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:33849)
0806 06:53:18.587639 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.587666 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:53:18.587970 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.587886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:50233)
0806 06:53:18.587934 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.587955 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:53:18.588116 19177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:53:18.595582 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19047
I20260806 06:53:18.595890 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:44385
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:38181
--webserver_port=40011
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:18.678334 19180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:18.678545 19180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:18.678570 19180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:18.679785 19180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:18.679832 19180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:18.679859 19180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:18.681401 19180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:44385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=40011
--tserver_master_addrs=127.3.247.190:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:18.681660 19180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:18.681974 19180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:18.687044 19185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.687124 19188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:18.687083 19186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:18.687402 19180 server_base.cc:1034] running on GCE node
I20260806 06:53:18.687613 19180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:18.687877 19180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:18.689029 19180 hybrid_clock.cc:648] HybridClock initialized: now 1785999198689027 us; error 40 us; skew 500 ppm
Aug 06 06:53:18 dist-test-slave-0kls krb5kdc[17740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:18.691505 19180 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:18.691954 19180 webserver.cc:460] Webserver started at http://127.3.247.131:40011/ using document root <none> and password file <none>
I20260806 06:53:18.692155 19180 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:18.692196 19180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:18.693476 19180 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:18.694044 19195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.694247 19180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:18.694329 19180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "98de812a7a504f1a9a51874829e7922c"
format_stamp: "Formatted at 2026-08-06 06:53:17 on dist-test-slave-0kls"
server_key: "84842bfb35c59e576dceb2389ec341d8"
server_key_iv: "ad8c2f38716091590374402605c6f787"
server_key_version: "encryptionkey@0"
I20260806 06:53:18.694646 19180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:18.702416 19180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:18.704223 19180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:18.704339 19180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:18.704509 19180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:18.704741 19180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:18.704771 19180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.704792 19180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:18.704805 19180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:18.710847 19180 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:44385
I20260806 06:53:18.710899 19308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:44385 every 8 connection(s)
I20260806 06:53:18.711266 19180 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:53:18.712482 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.712006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:51951)
0806 06:53:18.712184 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.712326 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":115,"threads_started":1}
W20260806 06:53:18.712919 19309 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:38181 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:38181: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
I20260806 06:53:18.713213 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.713111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:39905)
0806 06:53:18.713161 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.713187 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:53:18.713557 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:18.713470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:53163)
0806 06:53:18.713516 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:18.713542 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:53:18.713678 19309 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:53:18.721007 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19180
I20260806 06:53:19.520469 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:19.520205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:39201)
0806 06:53:19.520356 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:19.520402 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260806 06:53:19.588677 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:19.588462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:40077)
0806 06:53:19.588587 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:19.588626 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260806 06:53:19.714294 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:19.714074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:36401)
0806 06:53:19.714191 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:19.714233 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260806 06:53:20.087518 19316 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:20.087651 19316 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:20.087927 19316 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
Aug 06 06:53:20 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.130@KRBTEST.COM
Aug 06 06:53:20 dist-test-slave-0kls krb5kdc[17740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999198, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.131@KRBTEST.COM
I20260806 06:53:20.108937 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.088426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:37779 (local address 127.3.247.130:44461)
0806 06:53:20.088556 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:20.088563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:20.088588 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:53:20.089125 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:20.089129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:20.089188 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:20.089353 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:20.090304 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.091740 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.093339 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.094019 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.094148 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:20.105216 (+ 11068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:20.105223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:20.105223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:20.105238 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:20.107191 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107684 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:20.107687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:20.107687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:20.107784 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:20.108052 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:20.108053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:20.108053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:20.108306 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:20.108390 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:20.108680 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 06:53:20.108818 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":48,"threads_started":1}
I20260806 06:53:20.108927 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.088239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44461 (local address 127.3.247.129:37779)
0806 06:53:20.088391 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:20.088405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:20.088465 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:20.089563 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:20.089564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:20.089902 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:20.090179 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:20.090187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.091879 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.091880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:20.093219 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:20.093223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.094132 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.094133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:20.094288 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:20.103027 (+ 8739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:20.103041 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:20.105042 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:20.107315 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:20.107326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:20.107594 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:20.107855 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:20.107857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:20.107977 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:20.108385 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:20.108392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:20.108521 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:53:20.108629 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:53:20.108829 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":89,"thread_start_us":86,"threads_started":1}
I20260806 06:53:20.108949 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.088334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:48823 (local address 127.3.247.131:44385)
0806 06:53:20.088511 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:20.088524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:20.088562 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:53:20.089125 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:20.089129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:20.089188 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:20.089353 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:20.090311 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.091740 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.093339 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.094019 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.094144 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:20.105632 (+ 11488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:20.105636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:20.105637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:20.105649 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:20.107280 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107703 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:20.107707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:20.107707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:20.107800 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:20.108046 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:20.108047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:20.108047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:20.108303 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:20.108390 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:20.108680 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 06:53:20.108813 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":54,"threads_started":1}
I20260806 06:53:20.109052 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.088240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44385 (local address 127.3.247.129:48823)
0806 06:53:20.088407 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:20.088416 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:20.088465 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:20.089439 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:20.089442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:20.089902 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:20.090179 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:20.090187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.091888 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.091889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:20.093219 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:20.093223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:20.094129 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:20.094130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:20.094289 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:20.103027 (+ 8738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:20.103041 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:20.105489 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:20.107372 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107374 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:20.107374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:20.107618 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:20.107879 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:20.107880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:20.107880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:20.107980 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:20.108382 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:20.108389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:20.108521 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:53:20.108628 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:53:20.108912 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":197,"spinlock_wait_cycles":12032,"thread_start_us":108,"threads_started":1}
W20260806 06:53:20.110368 18931 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
W20260806 06:53:20.110468 18929 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:20.110543 18929 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:20.110653 19316 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:53:20.521200 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.521036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:54859)
0806 06:53:20.521104 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:20.521156 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:53:20.589375 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.589152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:46013)
0806 06:53:20.589285 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:20.589319 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260806 06:53:20.714977 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:20.714746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:34957)
0806 06:53:20.714872 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:20.714933 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 06:53:21.522120 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:21.521846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:56809)
0806 06:53:21.522019 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:21.522071 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260806 06:53:21.590164 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:21.589898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:57743)
0806 06:53:21.590073 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:21.590120 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260806 06:53:21.715735 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:21.715505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:49039)
0806 06:53:21.715639 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:21.715687 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260806 06:53:21.856510 19326 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:21.856611 19326 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:21.856853 19326 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
W20260806 06:53:21.857329 18929 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
W20260806 06:53:21.857399 18931 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:21.857468 18931 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:21.857550 19326 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:53:22.522944 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:22.522708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:41765)
0806 06:53:22.522852 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:22.522898 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260806 06:53:22.590824 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:22.590617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:33037)
0806 06:53:22.590745 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:22.590779 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260806 06:53:22.716528 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:22.716282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:49841)
0806 06:53:22.716448 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:22.716487 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260806 06:53:23.523662 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:23.523431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:41917)
0806 06:53:23.523581 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:23.523619 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260806 06:53:23.591512 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:23.591284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:35515)
0806 06:53:23.591422 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:23.591465 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260806 06:53:23.717248 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:23.717011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:51127)
0806 06:53:23.717144 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:23.717206 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260806 06:53:24.316119 19333 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:24.316215 19333 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:24.316407 19333 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
W20260806 06:53:24.316780 18931 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
W20260806 06:53:24.316911 18929 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:24.316991 18929 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:24.317080 19333 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:53:24.524478 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:24.524272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:41059)
0806 06:53:24.524395 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:24.524432 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260806 06:53:24.592190 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:24.591966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:40993)
0806 06:53:24.592102 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:24.592147 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260806 06:53:24.718001 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:24.717730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:50341)
0806 06:53:24.717918 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:24.717958 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":131,"threads_started":1}
I20260806 06:53:25.525157 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:25.524945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:55793)
0806 06:53:25.525070 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:25.525110 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260806 06:53:25.592935 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:25.592679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:41659)
0806 06:53:25.592843 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:25.592889 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260806 06:53:25.718742 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:25.718512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:32853)
0806 06:53:25.718652 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:25.718695 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260806 06:53:26.525873 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:26.525625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:59707)
0806 06:53:26.525739 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:26.525775 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260806 06:53:26.593696 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:26.593460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:52775)
0806 06:53:26.593609 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:26.593651 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260806 06:53:26.719535 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:26.719307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:43567)
0806 06:53:26.719448 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:26.719491 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260806 06:53:27.526572 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:27.526358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:48179)
0806 06:53:27.526487 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:27.526524 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260806 06:53:27.591156 19344 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:27.591245 19344 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:27.591447 19344 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
W20260806 06:53:27.591816 18929 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
W20260806 06:53:27.591928 18931 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:27.592000 18931 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:27.592110 19344 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:53:27.594369 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:27.594168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:40387)
0806 06:53:27.594283 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:27.594320 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260806 06:53:27.720257 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:27.720024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:57019)
0806 06:53:27.720168 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:27.720211 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260806 06:53:28.527513 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:28.527255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:34961)
0806 06:53:28.527411 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:28.527462 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260806 06:53:28.595126 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:28.594883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:39763)
0806 06:53:28.595038 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:28.595079 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260806 06:53:28.721017 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:28.720792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:35153)
0806 06:53:28.720930 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:28.720970 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260806 06:53:29.528358 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:29.528120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:40011)
0806 06:53:29.528267 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:29.528311 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260806 06:53:29.595907 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:29.595643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:51079)
0806 06:53:29.595799 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:29.595855 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260806 06:53:29.721994 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:29.721596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:44649)
0806 06:53:29.721740 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:29.721946 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260806 06:53:29.922386 19353 raft_consensus.cc:482] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:29.922495 19353 raft_consensus.cc:504] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98de812a7a504f1a9a51874829e7922c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44385 } } peers { permanent_uuid: "f1d5f9428f444930befe783c0508cb82" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44461 } } peers { permanent_uuid: "21ff454c71674d8898b5600a079d26bd" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 40353 } }
I20260806 06:53:29.922720 19353 leader_election.cc:288] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385), f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461)
W20260806 06:53:29.923126 18931 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1d5f9428f444930befe783c0508cb82 (127.3.247.130:44461): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
W20260806 06:53:29.923206 18929 leader_election.cc:341] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98de812a7a504f1a9a51874829e7922c (127.3.247.131:44385): Not found: Tablet not found: a8d547d7dae944a5a8bd2fffc4af27a2
I20260806 06:53:29.923297 18929 leader_election.cc:302] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21ff454c71674d8898b5600a079d26bd; no voters: 98de812a7a504f1a9a51874829e7922c, f1d5f9428f444930befe783c0508cb82
I20260806 06:53:29.923386 19353 raft_consensus.cc:2726] T a8d547d7dae944a5a8bd2fffc4af27a2 P 21ff454c71674d8898b5600a079d26bd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:53:30.529331 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:30.529071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:58243)
0806 06:53:30.529235 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:30.529283 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260806 06:53:30.596662 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:30.596416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:35025)
0806 06:53:30.596563 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:30.596609 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260806 06:53:30.722787 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:30.722586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:54901)
0806 06:53:30.722702 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:30.722740 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260806 06:53:31.530224 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:31.529984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:60859)
0806 06:53:31.530133 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:31.530176 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260806 06:53:31.597645 19358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:31.597407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:59261)
0806 06:53:31.597547 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:31.597596 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260806 06:53:31.723495 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:31.723276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:34125)
0806 06:53:31.723403 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:31.723447 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260806 06:53:32.530864 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:32.530657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:58187)
0806 06:53:32.530774 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:32.530817 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260806 06:53:32.598387 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:32.598178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:40727)
0806 06:53:32.598295 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:32.598338 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260806 06:53:32.724255 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:32.723997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:34789)
0806 06:53:32.724163 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:32.724209 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
I20260806 06:53:33.531852 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:33.531538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.129:37177)
0806 06:53:33.531712 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:33.531765 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260806 06:53:33.599179 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:33.598933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.130:34441)
0806 06:53:33.599083 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:33.599128 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":59,"threads_started":1}
I20260806 06:53:33.723026 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 18884
I20260806 06:53:33.724993 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:33.724796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38181 (local address 127.3.247.131:58923)
0806 06:53:33.724905 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:33.724950 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260806 06:53:33.728094 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19047
I20260806 06:53:33.732563 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19180
2026-08-06T06:53:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18238 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:53:33 dist-test-slave-0kls krb5kdc[19368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:53:33 dist-test-slave-0kls krb5kdc[19368](info): set up 2 sockets
Aug 06 06:53:33 dist-test-slave-0kls krb5kdc[19368](info): commencing operation
krb5kdc: starting...
W20260806 06:53:34.806008 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:53:34 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:53:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:53:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:53:34.930675 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:35.011027 19381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:35.011267 19381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:35.011305 19381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:35.012456 19381 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:53:35.012526 19381 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:53:35.012547 19381 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:53:35.012566 19381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:35.012599 19381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:35.012624 19381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:35.012650 19381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:35.012678 19381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:35.014246 19381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:35.014524 19381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:35.014748 19381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:35.019829 19386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.019863 19381 server_base.cc:1034] running on GCE node
W20260806 06:53:35.019850 19387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:35.019865 19389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.020318 19381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:35.020645 19381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:35.021836 19381 hybrid_clock.cc:648] HybridClock initialized: now 1785999215021827 us; error 44 us; skew 500 ppm
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:35.024188 19381 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:35.024616 19381 webserver.cc:460] Webserver started at http://127.3.247.190:35443/ using document root <none> and password file <none>
I20260806 06:53:35.024859 19381 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:35.024928 19381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:35.025104 19381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:35.026233 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "0dc9f773ebd941c48a28fba4e9302e7c"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "f7cb5d151614209c758dd53acb308186"
server_key_iv: "4f0c1be4c00cc577a635338dda0efa14"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.026597 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "0dc9f773ebd941c48a28fba4e9302e7c"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "f7cb5d151614209c758dd53acb308186"
server_key_iv: "4f0c1be4c00cc577a635338dda0efa14"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.027920 19381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:53:35.028668 19396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.028851 19381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:35.028970 19381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "0dc9f773ebd941c48a28fba4e9302e7c"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "f7cb5d151614209c758dd53acb308186"
server_key_iv: "4f0c1be4c00cc577a635338dda0efa14"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.029047 19381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:35.043035 19381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:35.044902 19381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:35.045096 19381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:35.049027 19381 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40639
I20260806 06:53:35.049079 19448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40639 every 8 connection(s)
I20260806 06:53:35.049433 19381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:35.050331 19449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:35.052485 19449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap starting.
I20260806 06:53:35.053025 19449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:35.053321 19449 log.cc:824] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:35.054001 19449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: No bootstrap required, opened a new log
I20260806 06:53:35.055609 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19381
I20260806 06:53:35.055709 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:53:35.055584 19449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:35.055759 19449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:35.055805 19449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Initialized, Role: FOLLOWER
I20260806 06:53:35.055935 4062 external_mini_cluster.cc:1428] Setting key dde1773f3c3e0ab65fa7ff10e11aabac
I20260806 06:53:35.055966 19449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:35.056067 19449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:35.056102 19449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:35.056156 19449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:35.057276 19449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:35.057363 19449 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc9f773ebd941c48a28fba4e9302e7c; no voters:
I20260806 06:53:35.057536 19449 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:53:35.057607 19454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:35.057749 19454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 LEADER]: Becoming Leader. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Running, Role: LEADER
I20260806 06:53:35.057829 19449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:35.057943 19454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:35.058357 19455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:53:35.058429 19455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:35.058646 19456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc9f773ebd941c48a28fba4e9302e7c. Latest consensus state: current_term: 1 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:53:35.058743 19456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:35.059178 19461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:35.059721 19461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:35.061337 19461 catalog_manager.cc:1347] Generated new cluster ID: 8a79d47c1d1b478d848c57fc0876f83d
I20260806 06:53:35.061390 19461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:35.067797 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.056399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.3.247.190:40639)
0806 06:53:35.056605 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.056615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.056652 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:53:35.056788 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.056798 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.056866 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.057037 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:35.057772 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.059399 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.060490 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.061033 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.061141 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.063949 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:35.063953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:35.063954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:35.063966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:35.066125 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.066534 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.066538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.066539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.066672 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.066871 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.066872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.066873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.067194 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:35.067348 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.067424 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.067584 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260806 06:53:35.105988 19461 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:53:35.106786 19461 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:35.113440 19461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Generated new TSK 0
I20260806 06:53:35.113672 19461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:53:35.140579 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:35.222410 19475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:35.222621 19475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:35.222645 19475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:35.223803 19475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:35.223845 19475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:35.223899 19475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:35.225425 19475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:35.225685 19475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:35.225953 19475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:35.230940 19480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.231104 19475 server_base.cc:1034] running on GCE node
W20260806 06:53:35.230988 19483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:35.230943 19481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.231410 19475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:35.231711 19475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:35.232916 19475 hybrid_clock.cc:648] HybridClock initialized: now 1785999215232900 us; error 30 us; skew 500 ppm
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:35.235359 19475 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:35.235801 19475 webserver.cc:460] Webserver started at http://127.3.247.129:46593/ using document root <none> and password file <none>
I20260806 06:53:35.236030 19475 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:35.236099 19475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:35.236239 19475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:35.237299 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9d431fed79572c94f67ae191c1bf8363"
server_key_iv: "5f3db5cca8d0ea0dc57989a3e2347675"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.237666 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9d431fed79572c94f67ae191c1bf8363"
server_key_iv: "5f3db5cca8d0ea0dc57989a3e2347675"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.239008 19475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:35.239769 19490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.240170 19475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:35.240309 19475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9d431fed79572c94f67ae191c1bf8363"
server_key_iv: "5f3db5cca8d0ea0dc57989a3e2347675"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.240412 19475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:35.252666 19475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:35.254714 19475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:35.254881 19475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:35.255143 19475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:35.255465 19475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:35.255498 19475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.255522 19475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:35.255534 19475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.261468 19475 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:36081
I20260806 06:53:35.261544 19603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:36081 every 8 connection(s)
I20260806 06:53:35.261976 19475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:35.265424 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19475
I20260806 06:53:35.265513 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:53:35.265729 4062 external_mini_cluster.cc:1428] Setting key b76935c7537d06bedc50cbbbeb95a949
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:35.272419 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.262741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:52895 (local address 127.3.247.190:40639)
0806 06:53:35.262806 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.262808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.263034 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:53:35.263586 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.263587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.263607 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.263657 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:35.264635 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.265284 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.266658 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.266877 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.266964 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.269526 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:35.269528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:35.269528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:35.269543 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:35.270988 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.271414 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.271415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.271416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.271478 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.271747 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.271748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.271748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.271870 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:35.271940 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.272239 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.272327 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:53:35.272650 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.262739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:52895)
0806 06:53:35.262904 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.262941 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.263465 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.263727 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.263732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.264329 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:35.264528 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.264533 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.265387 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.265388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.266561 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.266564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.266975 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.266976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.267156 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.267779 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:35.267791 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:35.269406 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:35.271085 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.271093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.271095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.271333 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.271556 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.271557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.271557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.271678 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.271943 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:35.271951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:35.272098 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.272204 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.272366 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260806 06:53:35.273149 19604 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:35.273244 19604 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:35.273440 19604 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:35.274044 19413 ts_manager.cc:194] Registered new tserver with Master: ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
I20260806 06:53:35.274745 19413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:52895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:53:35.290756 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:35.372076 19609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:35.372332 19609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:35.372371 19609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:35.373541 19609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:35.373631 19609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:35.373675 19609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:35.375197 19609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:35.375463 19609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:35.375694 19609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:35.380666 19614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.380860 19609 server_base.cc:1034] running on GCE node
W20260806 06:53:35.380657 19617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:35.380657 19615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.381197 19609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:35.381458 19609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:35.382615 19609 hybrid_clock.cc:648] HybridClock initialized: now 1785999215382600 us; error 30 us; skew 500 ppm
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:35.385135 19609 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:35.385571 19609 webserver.cc:460] Webserver started at http://127.3.247.130:43805/ using document root <none> and password file <none>
I20260806 06:53:35.385862 19609 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:35.385928 19609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:35.386078 19609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:35.387138 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "03006ce657134ddc92e4847d16a5f5a4"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "b7c26823d98dff0c0a8b17bf448077f6"
server_key_iv: "74a398b9a1ecc9d055ccfe90e428c4db"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.387519 19609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "03006ce657134ddc92e4847d16a5f5a4"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "b7c26823d98dff0c0a8b17bf448077f6"
server_key_iv: "74a398b9a1ecc9d055ccfe90e428c4db"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.389051 19609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:53:35.389930 19624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.390187 19609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:35.390301 19609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "03006ce657134ddc92e4847d16a5f5a4"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "b7c26823d98dff0c0a8b17bf448077f6"
server_key_iv: "74a398b9a1ecc9d055ccfe90e428c4db"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.390388 19609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:35.400183 19609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:35.402374 19609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:35.402551 19609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:35.402806 19609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:35.403149 19609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:35.403204 19609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.403239 19609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:35.403273 19609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.409040 19609 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:46573
I20260806 06:53:35.409080 19737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:46573 every 8 connection(s)
I20260806 06:53:35.409411 19609 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:35.415632 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19609
I20260806 06:53:35.415707 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:53:35.415949 4062 external_mini_cluster.cc:1428] Setting key 9de84209f3a7d52620a13d956eaa5ddc
I20260806 06:53:35.419004 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.410226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:51465 (local address 127.3.247.190:40639)
0806 06:53:35.410313 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.410316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.410485 (+ 169us) server_negotiation.cc:407] Connection header received
0806 06:53:35.410993 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.410994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.411005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.411059 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:35.411972 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.412473 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.413427 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.413623 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.413675 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.415772 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:35.415774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:35.415774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:35.415782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:35.417511 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.417965 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.417967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.417967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.418043 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.418354 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.418355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.418356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.418465 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:35.418522 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.418857 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.418927 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:53:35.419281 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.410186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:51465)
0806 06:53:35.410360 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.410397 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.410895 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.411112 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.411115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.411684 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:35.411897 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.411901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.412551 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.412552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.413354 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.413356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.413697 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.413697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.413822 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.414297 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:35.414305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:35.415646 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:35.417595 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.417601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.417603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.417896 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.418119 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.418120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.418120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.418284 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.418536 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:35.418542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:35.418716 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.418841 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.419023 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260806 06:53:35.419785 19738 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:35.419900 19738 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:35.420092 19738 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:35.420477 19413 ts_manager.cc:194] Registered new tserver with Master: 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573)
I20260806 06:53:35.421030 19413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:51465
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:53:35.440864 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:35.522900 19743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:35.523113 19743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:35.523137 19743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:35.524339 19743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:35.524387 19743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:35.524415 19743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:35.525982 19743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:35.526230 19743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:35.526506 19743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:35.532016 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:35.532016 19748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:35.532047 19751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:35.532300 19743 server_base.cc:1034] running on GCE node
I20260806 06:53:35.532573 19743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:35.532910 19743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:35.534088 19743 hybrid_clock.cc:648] HybridClock initialized: now 1785999215534084 us; error 50 us; skew 500 ppm
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:35.536684 19743 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:35.537207 19743 webserver.cc:460] Webserver started at http://127.3.247.131:46513/ using document root <none> and password file <none>
I20260806 06:53:35.537456 19743 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:35.537530 19743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:35.537681 19743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:53:35.538836 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "b1fe05102e9946f78c56119593cf51f9"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9865938a4000026f7c8c37b9f731ad8e"
server_key_iv: "7857e8d0087f55ed488a948b7a3f72b8"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.539216 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "b1fe05102e9946f78c56119593cf51f9"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9865938a4000026f7c8c37b9f731ad8e"
server_key_iv: "7857e8d0087f55ed488a948b7a3f72b8"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.540585 19743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:53:35.541424 19758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.541683 19743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:35.541819 19743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b1fe05102e9946f78c56119593cf51f9"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9865938a4000026f7c8c37b9f731ad8e"
server_key_iv: "7857e8d0087f55ed488a948b7a3f72b8"
server_key_version: "encryptionkey@0"
I20260806 06:53:35.541916 19743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:35.556242 19743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:35.558883 19743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:35.559063 19743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:35.559377 19743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:35.559725 19743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:35.559782 19743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.559815 19743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:35.559842 19743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:35.565820 19743 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:33645
I20260806 06:53:35.565869 19871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:33645 every 8 connection(s)
I20260806 06:53:35.566190 19743 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:53:35 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999215, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:35.575551 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.567046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:52905 (local address 127.3.247.190:40639)
0806 06:53:35.567128 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.567131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.567281 (+ 150us) server_negotiation.cc:407] Connection header received
0806 06:53:35.567760 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.567762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.567776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.567895 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:35.568868 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.569364 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.570414 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.570599 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.570656 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.572907 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:35.572908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:35.572908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:35.572915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:35.574229 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.574618 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.574618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.574619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.574668 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.574917 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.574919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.574919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.575071 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:35.575146 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.575400 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.575470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:53:35.575667 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.566990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:52905)
0806 06:53:35.567152 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.567190 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.567666 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.567951 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.567954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.568547 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:35.568742 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.568746 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.569460 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.569461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.570337 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.570339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.570675 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.570676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.570772 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.571326 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:35.571334 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:35.572787 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:35.574311 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.574317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.574319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.574550 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.574728 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:35.574728 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:35.574728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:35.574831 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:35.575141 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:35.575145 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:35.575261 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.575344 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.575469 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260806 06:53:35.576082 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19743
I20260806 06:53:35.576190 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:53:35.576272 19872 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:35.576386 19872 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:35.576399 4062 external_mini_cluster.cc:1428] Setting key b24fb9a06a2a284556a61d93dd1b87a4
I20260806 06:53:35.576601 19872 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:35.577008 19413 ts_manager.cc:194] Registered new tserver with Master: b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
I20260806 06:53:35.577369 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:35.577447 19413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:52905
I20260806 06:53:35.584785 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.579074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.3.247.190:40639)
0806 06:53:35.579115 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.579117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.579227 (+ 110us) server_negotiation.cc:407] Connection header received
0806 06:53:35.579288 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.579289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.579297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.579345 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:35.579865 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.580340 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.581059 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.581229 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.581279 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.582510 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:35.582511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:35.582512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:35.582519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:35.583770 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.584103 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.584104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.584104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.584166 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:35.584335 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:35.584335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:35.584335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:35.584492 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:35.584567 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.584658 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.584718 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:53:35.586581 19413 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:53:35.588167 19413 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:53:35.594817 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.3.247.130:46573)
0806 06:53:35.590736 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.590741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.591112 (+ 371us) server_negotiation.cc:407] Connection header received
0806 06:53:35.591163 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.591165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.591184 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.591259 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:35.591542 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.592221 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.593573 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.594414 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.594488 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.594549 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.594715 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.594749 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260806 06:53:35.594830 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:46573 (local address 127.0.0.1:54492)
0806 06:53:35.590761 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.590802 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.590903 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.591346 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.591350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.591354 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:35.591440 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.591442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.592319 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.592320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.593471 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.593474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.594514 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.594515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.594601 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.594618 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.594692 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.594745 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":21,"thread_start_us":78,"threads_started":1}
I20260806 06:53:35.595337 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.0.0.1:46286)
0806 06:53:35.591004 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.591014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.591050 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.591347 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.591350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.591352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:35.591454 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.591456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.592353 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.592354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.593603 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.593605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.595106 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.595107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.595167 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.595171 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.595219 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.595256 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":87,"thread_start_us":40,"threads_started":1}
I20260806 06:53:35.595623 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.3.247.129:36081)
0806 06:53:35.591123 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.591127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.591136 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:53:35.591169 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.591171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.591184 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.591259 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:35.591536 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.592231 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.593672 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.594414 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.594500 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.594549 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.595484 (+ 935us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.595536 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
I20260806 06:53:35.595676 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:33645 (local address 127.0.0.1:35336)
0806 06:53:35.590892 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.590912 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.591070 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.591347 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.591350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.591355 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:35.591450 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.591452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.593597 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.593598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.594452 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.594476 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.595443 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.595445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.595517 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.595520 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.595567 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.595600 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":29,"thread_start_us":53,"threads_started":1}
I20260806 06:53:35.595703 19672 tablet_service.cc:1467] Processing CreateTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 (DEFAULT_TABLE table=test-table [id=209fc40edfe9439e9423bb1fb7694210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:35.595890 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.590645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.3.247.131:33645)
0806 06:53:35.590864 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.590868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.591002 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:53:35.591169 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.591171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.591188 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.591260 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:35.592347 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.593471 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.594564 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.595294 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.595354 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.595411 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.595770 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.595816 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260806 06:53:35.596110 19672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c417ff9c4bb4a2482693d8144e2df55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:35.596587 19806 tablet_service.cc:1467] Processing CreateTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 (DEFAULT_TABLE table=test-table [id=209fc40edfe9439e9423bb1fb7694210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:35.596911 19806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c417ff9c4bb4a2482693d8144e2df55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:35.598320 19538 tablet_service.cc:1467] Processing CreateTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 (DEFAULT_TABLE table=test-table [id=209fc40edfe9439e9423bb1fb7694210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:35.598634 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c417ff9c4bb4a2482693d8144e2df55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:35.599656 19886 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Bootstrap starting.
I20260806 06:53:35.600227 19887 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Bootstrap starting.
I20260806 06:53:35.600453 19886 tablet_bootstrap.cc:654] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:35.600893 19887 tablet_bootstrap.cc:654] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:35.601131 19886 log.cc:824] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:35.601475 19887 log.cc:824] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:35.601959 19886 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: No bootstrap required, opened a new log
I20260806 06:53:35.602016 19886 ts_tablet_manager.cc:1397] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:35.602325 19887 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: No bootstrap required, opened a new log
I20260806 06:53:35.602380 19887 ts_tablet_manager.cc:1397] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:35.603403 19886 raft_consensus.cc:348] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.603578 19886 raft_consensus.cc:374] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:35.603626 19886 raft_consensus.cc:729] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03006ce657134ddc92e4847d16a5f5a4, State: Initialized, Role: FOLLOWER
I20260806 06:53:35.603742 19886 consensus_queue.cc:260] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.603763 19887 raft_consensus.cc:348] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.603968 19887 raft_consensus.cc:374] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:35.604017 19887 raft_consensus.cc:729] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fe05102e9946f78c56119593cf51f9, State: Initialized, Role: FOLLOWER
I20260806 06:53:35.604029 19886 ts_tablet_manager.cc:1428] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:35.604140 19738 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
I20260806 06:53:35.604128 19887 consensus_queue.cc:260] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.604413 19887 ts_tablet_manager.cc:1428] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:35.604418 19872 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
I20260806 06:53:35.604506 19889 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap starting.
I20260806 06:53:35.605288 19889 tablet_bootstrap.cc:654] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:35.605757 19889 log.cc:824] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:35.606438 19889 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: No bootstrap required, opened a new log
I20260806 06:53:35.606504 19889 ts_tablet_manager.cc:1397] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:53:35.607829 19889 raft_consensus.cc:348] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.607957 19889 raft_consensus.cc:374] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:35.608017 19889 raft_consensus.cc:729] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1a5e6ad9b34adbbb7bad568b7f43ff, State: Initialized, Role: FOLLOWER
I20260806 06:53:35.608135 19889 consensus_queue.cc:260] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.608418 19889 ts_tablet_manager.cc:1428] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:35.608431 19604 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
W20260806 06:53:35.659991 19739 tablet.cc:2369] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:53:35.762723 19605 tablet.cc:2369] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:35.801566 19891 raft_consensus.cc:482] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:35.801688 19891 raft_consensus.cc:504] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.802117 19891 leader_election.cc:288] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645), ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
I20260806 06:53:35.805099 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.802412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:47655 (local address 127.3.247.131:33645)
0806 06:53:35.802473 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.802476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.802527 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:53:35.802557 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.802559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.802569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.802622 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:35.802813 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.803305 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.804289 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.804701 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.804790 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.804801 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.804967 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.805015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:53:35.805105 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.802353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:33645 (local address 127.3.247.130:47655)
0806 06:53:35.802396 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.802408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.802454 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.802661 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.802662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.802664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:35.802734 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.802736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.803392 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.803393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.804131 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.804133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.804786 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.804787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.804889 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.804892 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.804962 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.805009 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 06:53:35.805397 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.802461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.3.247.130:49109)
0806 06:53:35.802641 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.802651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:35.802727 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:35.802866 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:35.802866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:35.802868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:35.802944 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.802945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.803563 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.803564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.804363 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:35.804365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.805099 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.805101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:35.805188 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.805192 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:53:35.805269 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:53:35.805320 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":136,"threads_started":1}
I20260806 06:53:35.805415 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.802569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:49109 (local address 127.3.247.129:36081)
0806 06:53:35.802612 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.802614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.802720 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:53:35.802782 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.802784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.802793 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.802865 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:35.803019 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.803481 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.804503 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.804982 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.805090 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.805099 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.805291 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.805345 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:53:35.805547 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c417ff9c4bb4a2482693d8144e2df55" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fe05102e9946f78c56119593cf51f9" is_pre_election: true
I20260806 06:53:35.805735 19826 raft_consensus.cc:2454] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 0.
I20260806 06:53:35.806185 19628 leader_election.cc:302] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9; no voters:
I20260806 06:53:35.806336 19891 raft_consensus.cc:2781] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:35.806293 19558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c417ff9c4bb4a2482693d8144e2df55" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" is_pre_election: true
I20260806 06:53:35.806422 19891 raft_consensus.cc:482] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:35.806442 19891 raft_consensus.cc:3037] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:35.806504 19558 raft_consensus.cc:2454] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 0.
I20260806 06:53:35.807271 19891 raft_consensus.cc:504] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.807408 19891 leader_election.cc:288] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 election: Requested vote from peers b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645), ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
I20260806 06:53:35.807572 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c417ff9c4bb4a2482693d8144e2df55" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fe05102e9946f78c56119593cf51f9"
I20260806 06:53:35.807641 19558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c417ff9c4bb4a2482693d8144e2df55" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
I20260806 06:53:35.807646 19826 raft_consensus.cc:3037] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:35.807718 19558 raft_consensus.cc:3037] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:35.808367 19826 raft_consensus.cc:2454] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 1.
I20260806 06:53:35.808452 19558 raft_consensus.cc:2454] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 1.
I20260806 06:53:35.808527 19628 leader_election.cc:302] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9; no voters:
I20260806 06:53:35.808614 19891 raft_consensus.cc:2781] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:35.808777 19891 raft_consensus.cc:686] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 03006ce657134ddc92e4847d16a5f5a4, State: Running, Role: LEADER
I20260806 06:53:35.808913 19891 consensus_queue.cc:237] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:35.809765 19412 catalog_manager.cc:5495] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "03006ce657134ddc92e4847d16a5f5a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } health_report { overall_health: HEALTHY } } }
W20260806 06:53:35.816705 19873 tablet.cc:2369] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:35.823153 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.820903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54498 (local address 127.3.247.130:46573)
0806 06:53:35.820953 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:35.820955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:35.820962 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:53:35.821041 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:35.821042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:35.821057 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:35.821112 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:35.821201 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.821652 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.822434 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:35.822629 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:35.822685 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:35.822814 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:35.822906 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:35.822965 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:35.823008 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:53:35.823025 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:53:35.824867 19826 raft_consensus.cc:1261] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Refusing update from remote peer 03006ce657134ddc92e4847d16a5f5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:35.824867 19558 raft_consensus.cc:1261] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Refusing update from remote peer 03006ce657134ddc92e4847d16a5f5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:35.825326 19891 consensus_queue.cc:1035] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:35.825470 19896 consensus_queue.cc:1035] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:35.828962 19901 mvcc.cc:205] Tried to move back new op lower bound from 7315452788015878144 to 7315452787953860608. Current Snapshot: MvccSnapshot[applied={T|T < 7315452788015878144}]
I20260806 06:53:35.829226 19900 mvcc.cc:205] Tried to move back new op lower bound from 7315452788015878144 to 7315452787953860608. Current Snapshot: MvccSnapshot[applied={T|T < 7315452788015878144 or (T in {7315452788015878144})}]
I20260806 06:53:35.830659 19412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:35.830775 19412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:35.831568 19412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Sending DeleteTablet for 3 replicas of tablet 3c417ff9c4bb4a2482693d8144e2df55
I20260806 06:53:35.831918 19672 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:35 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:54492
I20260806 06:53:35.831908 19538 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:35 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:46286
I20260806 06:53:35.832181 19908 tablet_replica.cc:330] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: stopping tablet replica
I20260806 06:53:35.832182 19806 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:35 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:35336
I20260806 06:53:35.832355 19908 raft_consensus.cc:2229] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:53:35.832377 19910 tablet_replica.cc:330] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: stopping tablet replica
I20260806 06:53:35.832450 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19475
I20260806 06:53:35.832464 19910 raft_consensus.cc:2229] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:35.832705 19908 raft_consensus.cc:2258] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:35.833326 19908 ts_tablet_manager.cc:1905] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:35.834146 19910 raft_consensus.cc:2258] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:35.835145 19908 ts_tablet_manager.cc:1918] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:35.835211 19908 log.cc:1196] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/3c417ff9c4bb4a2482693d8144e2df55
I20260806 06:53:35.835510 19908 ts_tablet_manager.cc:1939] T 3c417ff9c4bb4a2482693d8144e2df55 P 03006ce657134ddc92e4847d16a5f5a4: Deleting consensus metadata
I20260806 06:53:35.836041 19399 catalog_manager.cc:4841] TS 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): tablet 3c417ff9c4bb4a2482693d8144e2df55 (table test-table [id=209fc40edfe9439e9423bb1fb7694210]) successfully deleted
I20260806 06:53:35.836424 19910 ts_tablet_manager.cc:1905] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:35.838443 19910 ts_tablet_manager.cc:1918] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:35.838531 19910 log.cc:1196] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/3c417ff9c4bb4a2482693d8144e2df55
I20260806 06:53:35.838855 19910 ts_tablet_manager.cc:1939] T 3c417ff9c4bb4a2482693d8144e2df55 P b1fe05102e9946f78c56119593cf51f9: Deleting consensus metadata
I20260806 06:53:35.839314 19400 catalog_manager.cc:4841] TS b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): tablet 3c417ff9c4bb4a2482693d8144e2df55 (table test-table [id=209fc40edfe9439e9423bb1fb7694210]) successfully deleted
W20260806 06:53:35.840896 19400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:36081 (error 108)
I20260806 06:53:35.841140 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19609
I20260806 06:53:35.841429 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:35.841301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.0.0.1:46290)
0806 06:53:35.841355 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:35.841397 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:53:35.841523 19400 catalog_manager.cc:4584] TS ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c417ff9c4bb4a2482693d8144e2df55: Network error: Client connection negotiation failed: client connection to 127.3.247.129:36081: connect: Connection refused (error 111)
I20260806 06:53:35.846369 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19743
I20260806 06:53:35.852167 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19381
I20260806 06:53:55.857586 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--webserver_interface=127.3.247.190
--webserver_port=35443
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:53:55.941691 19912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:55.941929 19912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:55.941953 19912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:55.943079 19912 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:53:55.943123 19912 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:53:55.943136 19912 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:53:55.943148 19912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:53:55.943161 19912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:55.943176 19912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:55.943187 19912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:53:55.943199 19912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:53:55.944634 19912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=35443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:55.944840 19912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:55.945019 19912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:55.954226 19919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:55.954226 19921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:55.954319 19912 server_base.cc:1034] running on GCE node
W20260806 06:53:55.954226 19918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:55.954713 19912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:55.955061 19912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:55.956241 19912 hybrid_clock.cc:648] HybridClock initialized: now 1785999235956230 us; error 45 us; skew 500 ppm
Aug 06 06:53:55 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:55.958881 19912 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:53:55.959311 19912 webserver.cc:460] Webserver started at http://127.3.247.190:35443/ using document root <none> and password file <none>
I20260806 06:53:55.959513 19912 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:55.959568 19912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:55.960922 19912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:55.961547 19928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:55.961722 19912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:55.961838 19912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "0dc9f773ebd941c48a28fba4e9302e7c"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "f7cb5d151614209c758dd53acb308186"
server_key_iv: "4f0c1be4c00cc577a635338dda0efa14"
server_key_version: "encryptionkey@0"
I20260806 06:53:55.962262 19912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:55.975164 19912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:55.977291 19912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:55.977451 19912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:55.981611 19912 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40639
I20260806 06:53:55.981662 19980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40639 every 8 connection(s)
I20260806 06:53:55.982105 19912 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:53:55.982862 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19912
I20260806 06:53:55.983189 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:36081
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=46593
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:55.984360 19981 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:53:55.985211 19981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap starting.
I20260806 06:53:55.993762 19981 log.cc:824] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:55.996057 19981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:55.996443 19981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap complete.
I20260806 06:53:55.998955 19981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:55.999202 19981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Initialized, Role: FOLLOWER
I20260806 06:53:55.999385 19981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:55.999450 19981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:53:55.999507 19981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:53:55.999557 19981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:53:56.000604 19981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:56.000721 19981 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc9f773ebd941c48a28fba4e9302e7c; no voters:
I20260806 06:53:56.000939 19981 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:53:56.000982 19985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:53:56.001160 19985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 LEADER]: Becoming Leader. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Running, Role: LEADER
I20260806 06:53:56.001240 19981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:53:56.001317 19985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:53:56.001698 19987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:53:56.001899 19987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:56.003079 20001 catalog_manager.cc:1259] Loaded cluster ID: 8a79d47c1d1b478d848c57fc0876f83d
I20260806 06:53:56.003165 20001 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: loading cluster ID for follower catalog manager: success
I20260806 06:53:56.001699 19985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc9f773ebd941c48a28fba4e9302e7c. Latest consensus state: current_term: 2 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:53:56.003312 19985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:53:56.004451 20001 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: acquiring CA information for follower catalog manager: success
I20260806 06:53:56.004890 20001 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:53:56.005956 19994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:53:56.006421 19994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:53:56.006505 19994 catalog_manager.cc:1259] Loaded cluster ID: 8a79d47c1d1b478d848c57fc0876f83d
I20260806 06:53:56.006527 19994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:53:56.006681 19994 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:53:56.006793 19994 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Loaded TSK: 0
I20260806 06:53:56.019049 19994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Generated new TSK 1
I20260806 06:53:56.019193 19994 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:53:56.083560 19983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:56.083788 19983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:56.083813 19983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:56.085064 19983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:56.085114 19983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:56.085143 19983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:53:56.086759 19983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:36081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=46593
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:56.087019 19983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:56.087291 19983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:56.093406 20008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:56.093622 20011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:56.093430 20009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:56.093761 19983 server_base.cc:1034] running on GCE node
I20260806 06:53:56.094009 19983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:56.094280 19983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:56.095448 19983 hybrid_clock.cc:648] HybridClock initialized: now 1785999236095433 us; error 32 us; skew 500 ppm
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:56.098062 19983 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:53:56.098531 19983 webserver.cc:460] Webserver started at http://127.3.247.129:46593/ using document root <none> and password file <none>
I20260806 06:53:56.098783 19983 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:56.098846 19983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:56.100319 19983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:56.101130 20018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.101303 19983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:56.101389 19983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9d431fed79572c94f67ae191c1bf8363"
server_key_iv: "5f3db5cca8d0ea0dc57989a3e2347675"
server_key_version: "encryptionkey@0"
I20260806 06:53:56.101778 19983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:56.138429 19983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:56.141042 19983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:56.141196 19983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:56.141464 19983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:56.142010 20025 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:53:56.143504 19983 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:53:56.143564 19983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:53:56.143597 19983 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:53:56.144280 19983 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:53:56.144325 19983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:53:56.144372 20025 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap starting.
I20260806 06:53:56.152129 19983 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:36081
I20260806 06:53:56.152372 20132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:36081 every 8 connection(s)
I20260806 06:53:56.152674 19983 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:53:56.156209 20025 log.cc:824] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:56.158631 20025 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:53:56.158969 20025 tablet_bootstrap.cc:492] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap complete.
I20260806 06:53:56.159663 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 19983
I20260806 06:53:56.159955 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:46573
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=43805
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:53:56.160111 20025 ts_tablet_manager.cc:1397] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:56.161195 20025 raft_consensus.cc:348] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:56.161492 20025 raft_consensus.cc:729] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1a5e6ad9b34adbbb7bad568b7f43ff, State: Initialized, Role: FOLLOWER
I20260806 06:53:56.161671 20025 consensus_queue.cc:260] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } } peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } }
I20260806 06:53:56.161993 20025 ts_tablet_manager.cc:1428] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260806 06:53:56.165638 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.153457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:60419)
0806 06:53:56.153668 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.153764 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.154599 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.155040 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.155045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.155528 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:56.155851 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.155858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.157261 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.157262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.158164 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.158167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.158945 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.158946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.159084 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.159658 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:56.159668 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:56.161682 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:56.163839 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.163847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.163849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.164094 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.164391 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.164392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.164392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.164507 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.164947 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:56.164954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:56.165123 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.165240 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.165428 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260806 06:53:56.165637 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.153807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:60419 (local address 127.3.247.190:40639)
0806 06:53:56.153966 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.153975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.154021 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:53:56.154805 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.154810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.154853 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.154960 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.155970 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.157137 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.158260 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.158846 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.158962 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.161821 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.161825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.161826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.161840 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.163724 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.164199 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.164203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.164203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.164310 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.164575 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.164576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.164576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.164867 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.164970 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.165298 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.165435 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260806 06:53:56.167243 20133 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:56.167347 20133 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:56.167555 20133 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:56.168329 19945 ts_manager.cc:194] Registered new tserver with Master: ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
I20260806 06:53:56.169391 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:60419
I20260806 06:53:56.170341 20133 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999235, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.129@KRBTEST.COM
I20260806 06:53:56.178035 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.168987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.0.0.1:60876)
0806 06:53:56.169321 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.169364 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.169451 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.169837 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.169843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.170261 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:56.170336 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.170339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.171382 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.171384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.172330 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.172332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.172810 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.172811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.172888 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.173387 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:56.173393 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:56.174929 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:56.176763 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.176771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.176773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.176985 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.177230 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.177231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.177231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.177293 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.177603 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:56.177610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:56.177667 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.177772 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.177928 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":100,"threads_started":1}
I20260806 06:53:56.178035 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.169055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.3.247.129:36081)
0806 06:53:56.169250 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.169255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.169449 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:53:56.169649 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.169652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.169668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.169733 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.170523 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.171302 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.172429 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.172727 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.172795 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.175024 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.175032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.175033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.175040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.176654 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.177093 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.177096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.177096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.177158 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.177365 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.177366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.177366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.177528 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.177624 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.177826 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.177927 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260806 06:53:56.178745 20067 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c417ff9c4bb4a2482693d8144e2df55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:35 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:60876
I20260806 06:53:56.179280 20142 tablet_replica.cc:330] stopping tablet replica
I20260806 06:53:56.179395 20142 raft_consensus.cc:2229] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:56.179466 20142 raft_consensus.cc:2258] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:56.179836 20142 ts_tablet_manager.cc:1905] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:56.182601 20142 ts_tablet_manager.cc:1918] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:56.182694 20142 log.cc:1196] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal/wals/3c417ff9c4bb4a2482693d8144e2df55
I20260806 06:53:56.182930 20142 ts_tablet_manager.cc:1939] T 3c417ff9c4bb4a2482693d8144e2df55 P ac1a5e6ad9b34adbbb7bad568b7f43ff: Deleting consensus metadata
I20260806 06:53:56.183518 19932 catalog_manager.cc:4841] TS ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081): tablet 3c417ff9c4bb4a2482693d8144e2df55 (table test-table [id=209fc40edfe9439e9423bb1fb7694210]) successfully deleted
W20260806 06:53:56.247812 20138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:56.248034 20138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:56.248059 20138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:56.249188 20138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:56.249225 20138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:56.249253 20138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:53:56.250783 20138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:46573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=43805
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:56.250993 20138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:56.251227 20138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:56.256260 20147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:56.256300 20150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:56.256294 20138 server_base.cc:1034] running on GCE node
W20260806 06:53:56.256501 20148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:56.256760 20138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:56.257043 20138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:56.258215 20138 hybrid_clock.cc:648] HybridClock initialized: now 1785999236258199 us; error 27 us; skew 500 ppm
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:56.260689 20138 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:53:56.261081 20138 webserver.cc:460] Webserver started at http://127.3.247.130:43805/ using document root <none> and password file <none>
I20260806 06:53:56.261287 20138 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:56.261340 20138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:56.262718 20138 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:56.263249 20157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.263419 20138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:53:56.263504 20138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "03006ce657134ddc92e4847d16a5f5a4"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "b7c26823d98dff0c0a8b17bf448077f6"
server_key_iv: "74a398b9a1ecc9d055ccfe90e428c4db"
server_key_version: "encryptionkey@0"
I20260806 06:53:56.263831 20138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:56.286015 20138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:56.288446 20138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:56.288565 20138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:56.288838 20138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:56.289141 20138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:56.289175 20138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.289198 20138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:56.289211 20138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.295217 20138 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:46573
I20260806 06:53:56.295296 20270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:46573 every 8 connection(s)
I20260806 06:53:56.295635 20138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:53:56.295934 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20138
I20260806 06:53:56.296353 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:33645
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=46513
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:56.306638 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.296438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:52969 (local address 127.3.247.190:40639)
0806 06:53:56.296482 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.296484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.296850 (+ 366us) server_negotiation.cc:407] Connection header received
0806 06:53:56.297810 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.297811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.297824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.297870 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.298527 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.299038 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.300051 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.300281 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.300359 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.303041 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.303042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.303043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.303051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.304725 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.305366 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.305368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.305368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.305463 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.305904 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.305905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.305905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.306029 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.306088 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.306476 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.306558 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:53:56.307304 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.296446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:52969)
0806 06:53:56.296632 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.296693 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.297657 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.297944 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.297948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.298228 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:56.298421 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.298426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.299143 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.299144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.299957 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.299961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.300367 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.300368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.300508 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.301129 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:56.301138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:56.302927 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:56.304840 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.304853 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.304856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.305256 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.305567 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.305568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.305569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.305827 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.306102 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:56.306109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:56.306319 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.306463 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.306984 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260806 06:53:56.307879 20271 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:56.308003 20271 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:56.308197 20271 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:56.308586 19945 ts_manager.cc:194] Registered new tserver with Master: 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573)
I20260806 06:53:56.308995 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:52969
W20260806 06:53:56.384035 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:53:56.384291 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:53:56.384332 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:53:56.385524 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:53:56.385586 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:53:56.385629 20274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:53:56.387147 20274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:33645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=46513
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:53:56.387423 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:53:56.387660 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:53:56.392776 20280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:56.392830 20282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:53:56.392788 20279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:53:56.392936 20274 server_base.cc:1034] running on GCE node
I20260806 06:53:56.393196 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:53:56.393399 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:53:56.394536 20274 hybrid_clock.cc:648] HybridClock initialized: now 1785999236394522 us; error 28 us; skew 500 ppm
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:53:56.396987 20274 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:53:56.397441 20274 webserver.cc:460] Webserver started at http://127.3.247.131:46513/ using document root <none> and password file <none>
I20260806 06:53:56.397670 20274 fs_manager.cc:359] Metadata directory not provided
I20260806 06:53:56.397754 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:53:56.399184 20274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:56.400005 20289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.400378 20274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:53:56.400619 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b1fe05102e9946f78c56119593cf51f9"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9865938a4000026f7c8c37b9f731ad8e"
server_key_iv: "7857e8d0087f55ed488a948b7a3f72b8"
server_key_version: "encryptionkey@0"
I20260806 06:53:56.401100 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:53:56.428050 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:53:56.430052 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:53:56.430240 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:53:56.430491 20274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:53:56.430814 20274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:53:56.430871 20274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.430903 20274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:53:56.430948 20274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:53:56.437095 20274 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:33645
I20260806 06:53:56.437165 20402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:33645 every 8 connection(s)
I20260806 06:53:56.437521 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:53:56.442458 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20274
Aug 06 06:53:56 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999236, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:53:56.448050 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.438314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:52961 (local address 127.3.247.190:40639)
0806 06:53:56.438404 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.438406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.438589 (+ 183us) server_negotiation.cc:407] Connection header received
0806 06:53:56.439109 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.439110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.439121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.439177 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.440123 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.440647 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.441621 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.441887 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.441944 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.444440 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.444441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.444442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.444450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.446262 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.446825 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.446826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.446826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.446882 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.447221 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.447223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.447223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.447358 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.447514 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.447826 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.447935 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:53:56.448233 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.438278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:52961)
0806 06:53:56.438463 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.438500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.438977 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.439240 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.439243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.439815 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:53:56.440014 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.440019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.440743 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.440744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.441541 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.441543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.441952 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.441953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.442057 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.442749 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:53:56.442760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:53:56.444341 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:53:56.446345 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.446356 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.446359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.446743 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.446955 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:53:56.446956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:53:56.446956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:53:56.447125 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:53:56.447517 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:53:56.447523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:53:56.447660 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.447792 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.447980 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260806 06:53:56.448830 20403 heartbeater.cc:347] Connected to a master server at 127.3.247.190:40639
I20260806 06:53:56.448948 20403 heartbeater.cc:464] Registering TS with master...
I20260806 06:53:56.449141 20403 heartbeater.cc:511] Master 127.3.247.190:40639 requested a full tablet report, sending...
I20260806 06:53:56.449617 19945 ts_manager.cc:194] Registered new tserver with Master: b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
I20260806 06:53:56.450177 19945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:52961
I20260806 06:53:56.450531 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.443055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.3.247.190:40639)
0806 06:53:56.443200 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.443202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.443221 (+ 19us) server_negotiation.cc:407] Connection header received
0806 06:53:56.443246 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.443247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.443259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.443339 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.443949 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.444577 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.445630 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.446002 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.446087 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.447461 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.447462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.447463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.447470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.449258 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.449668 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.449669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.449670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.449738 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.449971 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.449971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.449972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.450100 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.450282 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.450327 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.450420 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260806 06:53:56.451385 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:53:56.454319 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.451747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.3.247.190:40639)
0806 06:53:56.451916 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.451918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.451980 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:53:56.452070 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.452070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.452099 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.452155 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:56.452253 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.452713 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.453518 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.453739 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.453839 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.454029 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:56.454081 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 06:53:56.454242 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55188: token expired
Metrics: {"server-negotiator.queue_time_us":148}
W20260806 06:53:56.454373 20407 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55188: token expired
W20260806 06:53:56.454489 19876 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.247.190:40639)
I20260806 06:53:56.460798 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.454767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.3.247.190:40639)
0806 06:53:56.454819 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.454821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.454877 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:53:56.454947 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.454948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.454956 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.455003 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:53:56.455498 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.455941 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.456648 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.456846 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.456930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.458224 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:53:56.458225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:53:56.458225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:53:56.458233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:53:56.459574 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.459975 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.459976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.459977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.460040 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:53:56.460265 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:53:56.460266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:53:56.460266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:53:56.460383 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:53:56.460487 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.460599 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.460690 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:53:56.462191 19945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:53:56.463816 19945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:53:56.466100 20067 tablet_service.cc:1467] Processing CreateTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef (DEFAULT_TABLE table=test-table [id=b846e2ecd13a42a2819289a3afece058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:56.466315 20067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6eb6674b0fa44eabdf711e9aaad0aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:56.467969 20025 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap starting.
I20260806 06:53:56.468313 20025 tablet_bootstrap.cc:654] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:56.469002 20025 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: No bootstrap required, opened a new log
I20260806 06:53:56.469060 20025 ts_tablet_manager.cc:1397] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:56.469401 20025 raft_consensus.cc:348] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.469475 20025 raft_consensus.cc:374] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:56.469543 20025 raft_consensus.cc:729] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1a5e6ad9b34adbbb7bad568b7f43ff, State: Initialized, Role: FOLLOWER
I20260806 06:53:56.469601 20025 consensus_queue.cc:260] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.469720 20025 ts_tablet_manager.cc:1428] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:53:56.472272 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.465758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:46573 (local address 127.0.0.1:38704)
0806 06:53:56.468347 (+ 2589us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.468359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.468470 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.468703 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.468704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.468706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:56.468754 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.468756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.470197 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.470198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.471242 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.471244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.471999 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.472000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.472068 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.472088 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.472146 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.472195 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2573,"thread_start_us":2531,"threads_started":1}
I20260806 06:53:56.472325 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.465974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.3.247.130:46573)
0806 06:53:56.466238 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.466244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.468436 (+ 2192us) server_negotiation.cc:407] Connection header received
0806 06:53:56.468586 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.468589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.468608 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.468697 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:56.468818 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.470135 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.471314 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.471917 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.471976 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.472016 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.472192 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.472233 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260806 06:53:56.472368 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.465739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:33645 (local address 127.0.0.1:59268)
0806 06:53:56.465840 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.465851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.465891 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.468431 (+ 2540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.468432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.468434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:56.468496 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.468498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.469973 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.469974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.471240 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.471243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.472060 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.472061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.472116 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.472143 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.472207 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.472255 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":27}
I20260806 06:53:56.472813 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.468028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.3.247.131:33645)
0806 06:53:56.468232 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.468237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.468251 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:53:56.468295 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.468298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.468315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.468569 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:56.468669 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.469869 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.471390 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.471936 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.472057 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.472130 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.472222 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.472263 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":123,"threads_started":1}
I20260806 06:53:56.473117 20205 tablet_service.cc:1467] Processing CreateTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef (DEFAULT_TABLE table=test-table [id=b846e2ecd13a42a2819289a3afece058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:56.473500 20205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6eb6674b0fa44eabdf711e9aaad0aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:56.473479 20337 tablet_service.cc:1467] Processing CreateTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef (DEFAULT_TABLE table=test-table [id=b846e2ecd13a42a2819289a3afece058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:53:56.473861 20337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6eb6674b0fa44eabdf711e9aaad0aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:53:56.476521 20413 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Bootstrap starting.
I20260806 06:53:56.476533 20414 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Bootstrap starting.
I20260806 06:53:56.477155 20414 tablet_bootstrap.cc:654] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:56.477147 20413 tablet_bootstrap.cc:654] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Neither blocks nor log segments found. Creating new log.
I20260806 06:53:56.477792 20413 log.cc:824] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:56.477831 20414 log.cc:824] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:53:56.478616 20413 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: No bootstrap required, opened a new log
I20260806 06:53:56.478674 20413 ts_tablet_manager.cc:1397] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:56.478690 20414 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: No bootstrap required, opened a new log
I20260806 06:53:56.478756 20414 ts_tablet_manager.cc:1397] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:56.480173 20414 raft_consensus.cc:348] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.480151 20413 raft_consensus.cc:348] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.480335 20413 raft_consensus.cc:374] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:56.480338 20414 raft_consensus.cc:374] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:53:56.480386 20414 raft_consensus.cc:729] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fe05102e9946f78c56119593cf51f9, State: Initialized, Role: FOLLOWER
I20260806 06:53:56.480386 20413 raft_consensus.cc:729] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03006ce657134ddc92e4847d16a5f5a4, State: Initialized, Role: FOLLOWER
I20260806 06:53:56.480508 20414 consensus_queue.cc:260] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.480531 20413 consensus_queue.cc:260] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.480827 20414 ts_tablet_manager.cc:1428] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:56.480825 20271 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
I20260806 06:53:56.480831 20413 ts_tablet_manager.cc:1428] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:53:56.480947 20403 heartbeater.cc:503] Master 127.3.247.190:40639 was elected leader, sending a full tablet report...
I20260806 06:53:56.518930 20418 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:53:56.519057 20418 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.519366 20418 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645), ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
I20260806 06:53:56.523362 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.519682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:53051 (local address 127.3.247.131:33645)
0806 06:53:56.519748 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.519751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.519763 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:53:56.519792 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.519794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.519803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.519928 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:56.520088 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.520866 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.522359 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.522919 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.523003 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.523014 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.523254 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.523296 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:53:56.523363 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.519816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:55469 (local address 127.3.247.129:36081)
0806 06:53:56.519918 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.519920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.520013 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:53:56.520078 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.520079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.520108 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.520180 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:53:56.520294 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.521012 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.522359 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.522987 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.523055 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.523119 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.523245 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.523288 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:53:56.523375 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.519605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:33645 (local address 127.3.247.130:53051)
0806 06:53:56.519667 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.519680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.519726 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.519934 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.519935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.519937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:56.520013 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.520015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.520951 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.520952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.522247 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.522250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.523053 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.523055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.523148 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.523151 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.523222 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.523274 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260806 06:53:56.523381 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.519736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.3.247.130:55469)
0806 06:53:56.519933 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.519954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:53:56.520008 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:53:56.520189 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:53:56.520189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:53:56.520191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:53:56.520240 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.520241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.521081 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.521082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.522247 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:53:56.522250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.523069 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.523070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:53:56.523146 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.523151 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:53:56.523226 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:53:56.523274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":18,"thread_start_us":120,"threads_started":1}
I20260806 06:53:56.523715 20087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6eb6674b0fa44eabdf711e9aaad0aef" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" is_pre_election: true
I20260806 06:53:56.523769 20357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6eb6674b0fa44eabdf711e9aaad0aef" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fe05102e9946f78c56119593cf51f9" is_pre_election: true
I20260806 06:53:56.523895 20087 raft_consensus.cc:2454] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 0.
I20260806 06:53:56.523904 20357 raft_consensus.cc:2454] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 0.
I20260806 06:53:56.524165 20161 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9; no voters:
I20260806 06:53:56.524323 20418 raft_consensus.cc:2781] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:53:56.524379 20418 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:53:56.524410 20418 raft_consensus.cc:3037] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:56.525147 20418 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.525302 20418 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 election: Requested vote from peers b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645), ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081)
I20260806 06:53:56.525478 20357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6eb6674b0fa44eabdf711e9aaad0aef" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1fe05102e9946f78c56119593cf51f9"
I20260806 06:53:56.525478 20087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6eb6674b0fa44eabdf711e9aaad0aef" candidate_uuid: "03006ce657134ddc92e4847d16a5f5a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
I20260806 06:53:56.525563 20087 raft_consensus.cc:3037] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:56.525563 20357 raft_consensus.cc:3037] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:53:56.526487 20087 raft_consensus.cc:2454] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 1.
I20260806 06:53:56.526487 20357 raft_consensus.cc:2454] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03006ce657134ddc92e4847d16a5f5a4 in term 1.
I20260806 06:53:56.526670 20161 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03006ce657134ddc92e4847d16a5f5a4, ac1a5e6ad9b34adbbb7bad568b7f43ff; no voters:
I20260806 06:53:56.526777 20418 raft_consensus.cc:2781] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:53:56.526925 20418 raft_consensus.cc:686] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 03006ce657134ddc92e4847d16a5f5a4, State: Running, Role: LEADER
I20260806 06:53:56.527024 20418 consensus_queue.cc:237] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:53:56.527859 19945 catalog_manager.cc:5495] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130). New cstate: current_term: 1 leader_uuid: "03006ce657134ddc92e4847d16a5f5a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260806 06:53:56.543391 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.541205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.3.247.130:46573)
0806 06:53:56.541255 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:53:56.541257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:53:56.541265 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:53:56.541300 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:53:56.541300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:53:56.541319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:53:56.541377 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:53:56.541475 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.542025 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.542701 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:53:56.542891 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:53:56.542939 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:53:56.543050 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:53:56.543162 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:53:56.543217 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:53:56.543245 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:53:56.543260 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:53:56.545264 20087 raft_consensus.cc:1261] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Refusing update from remote peer 03006ce657134ddc92e4847d16a5f5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:56.545264 20357 raft_consensus.cc:1261] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Refusing update from remote peer 03006ce657134ddc92e4847d16a5f5a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:53:56.545656 20418 consensus_queue.cc:1035] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:53:56.545850 20420 consensus_queue.cc:1035] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:53:56.546154 20272 tablet.cc:2369] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:53:56.550359 19945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:56.550484 19945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:53:56.551203 19945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Sending DeleteTablet for 3 replicas of tablet f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:53:56.551537 20067 tablet_service.cc:1514] Processing DeleteTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:56 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:60876
I20260806 06:53:56.551519 20337 tablet_service.cc:1514] Processing DeleteTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:56 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:59268
I20260806 06:53:56.551564 20205 tablet_service.cc:1514] Processing DeleteTablet for tablet f6eb6674b0fa44eabdf711e9aaad0aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:53:56 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:38704
I20260806 06:53:56.551641 20142 tablet_replica.cc:330] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: stopping tablet replica
I20260806 06:53:56.551697 20142 raft_consensus.cc:2229] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:56.551753 20142 raft_consensus.cc:2258] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:56.551868 20433 tablet_replica.cc:330] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: stopping tablet replica
I20260806 06:53:56.551877 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19983
I20260806 06:53:56.551991 20433 raft_consensus.cc:2229] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:53:56.552088 20433 raft_consensus.cc:2258] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:56.553467 20433 ts_tablet_manager.cc:1905] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:56.553771 20432 tablet_replica.cc:330] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: stopping tablet replica
I20260806 06:53:56.553925 20432 raft_consensus.cc:2229] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:53:56.554051 20432 raft_consensus.cc:2258] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:53:56.554436 20432 ts_tablet_manager.cc:1905] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:53:56.555506 20433 ts_tablet_manager.cc:1918] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:56.555584 20433 log.cc:1196] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:53:56.555840 20432 ts_tablet_manager.cc:1918] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:53:56.555878 20433 ts_tablet_manager.cc:1939] T f6eb6674b0fa44eabdf711e9aaad0aef P 03006ce657134ddc92e4847d16a5f5a4: Deleting consensus metadata
I20260806 06:53:56.555922 20432 log.cc:1196] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:53:56.556236 20432 ts_tablet_manager.cc:1939] T f6eb6674b0fa44eabdf711e9aaad0aef P b1fe05102e9946f78c56119593cf51f9: Deleting consensus metadata
I20260806 06:53:56.556282 19931 catalog_manager.cc:4841] TS 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): tablet f6eb6674b0fa44eabdf711e9aaad0aef (table test-table [id=b846e2ecd13a42a2819289a3afece058]) successfully deleted
I20260806 06:53:56.556627 19932 catalog_manager.cc:4841] TS b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): tablet f6eb6674b0fa44eabdf711e9aaad0aef (table test-table [id=b846e2ecd13a42a2819289a3afece058]) successfully deleted
W20260806 06:53:56.557893 19932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:36081 (error 108)
I20260806 06:53:56.558184 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 20138
I20260806 06:53:56.558368 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:53:56.558228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:36081 (local address 127.0.0.1:60892)
0806 06:53:56.558290 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:53:56.558329 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 06:53:56.559631 19932 catalog_manager.cc:4584] TS ac1a5e6ad9b34adbbb7bad568b7f43ff (127.3.247.129:36081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f6eb6674b0fa44eabdf711e9aaad0aef: Network error: Client connection negotiation failed: client connection to 127.3.247.129:36081: connect: Connection refused (error 111)
I20260806 06:53:56.563171 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 20274
I20260806 06:53:56.567751 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 19912
I20260806 06:54:16.573302 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--webserver_interface=127.3.247.190
--webserver_port=35443
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:16.656627 20435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:16.656863 20435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:16.656888 20435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:16.658208 20435 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:54:16.658259 20435 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:54:16.658272 20435 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:54:16.658284 20435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:16.658298 20435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:16.658311 20435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:16.658324 20435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:16.658375 20435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:16.659941 20435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=35443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:16.660233 20435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:16.660502 20435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:16.666155 20443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:16.666162 20440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:16.666256 20435 server_base.cc:1034] running on GCE node
W20260806 06:54:16.666194 20441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:16.666678 20435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:16.667019 20435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:16.668208 20435 hybrid_clock.cc:648] HybridClock initialized: now 1785999256668188 us; error 38 us; skew 500 ppm
Aug 06 06:54:16 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:16.675289 20435 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:16.675882 20435 webserver.cc:460] Webserver started at http://127.3.247.190:35443/ using document root <none> and password file <none>
I20260806 06:54:16.676148 20435 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:16.676231 20435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:16.678012 20435 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:16.678861 20450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:16.679051 20435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:16.679162 20435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "0dc9f773ebd941c48a28fba4e9302e7c"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "f7cb5d151614209c758dd53acb308186"
server_key_iv: "4f0c1be4c00cc577a635338dda0efa14"
server_key_version: "encryptionkey@0"
I20260806 06:54:16.679584 20435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:16.697562 20435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:16.700249 20435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:16.700429 20435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:16.704618 20435 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40639
I20260806 06:54:16.704658 20502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40639 every 8 connection(s)
I20260806 06:54:16.705086 20435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:16.706597 20503 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:54:16.707281 20503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap starting.
I20260806 06:54:16.708667 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20435
I20260806 06:54:16.708950 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:36081
--local_ip_for_outbound_sockets=127.3.247.129
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=46593
--webserver_interface=127.3.247.129
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:54:16.716437 20503 log.cc:824] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:16.718832 20503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:54:16.719174 20503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Bootstrap complete.
I20260806 06:54:16.721436 20503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:54:16.721664 20503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Initialized, Role: FOLLOWER
I20260806 06:54:16.721858 20503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:54:16.721932 20503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:16.721974 20503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:16.722018 20503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:54:16.723074 20503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:54:16.723192 20503 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc9f773ebd941c48a28fba4e9302e7c; no voters:
I20260806 06:54:16.723402 20503 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:54:16.723441 20507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:54:16.723568 20507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [term 3 LEADER]: Becoming Leader. State: Replica: 0dc9f773ebd941c48a28fba4e9302e7c, State: Running, Role: LEADER
I20260806 06:54:16.723654 20507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } }
I20260806 06:54:16.723681 20503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:16.723938 20507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc9f773ebd941c48a28fba4e9302e7c. Latest consensus state: current_term: 3 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:54:16.724004 20507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:16.724107 20509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc9f773ebd941c48a28fba4e9302e7c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40639 } } }
I20260806 06:54:16.724252 20509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:16.724318 20511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:16.725152 20511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:16.725266 20511 catalog_manager.cc:1259] Loaded cluster ID: 8a79d47c1d1b478d848c57fc0876f83d
I20260806 06:54:16.725306 20511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:16.726459 20511 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:16.726637 20511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Loaded TSK: 0
I20260806 06:54:16.726686 20511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dc9f773ebd941c48a28fba4e9302e7c: Loaded TSK: 1
F20260806 06:54:16.726912 20511 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbd2694e9ad google::LogMessage::Fail() at ??:0
@ 0x7fbd2695188a google::LogMessage::SendToLog() at ??:0
@ 0x7fbd2694e4de google::LogMessage::Flush() at ??:0
@ 0x7fbd2694fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbd290bb104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbd290b8d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbd2e8a81cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbd2e8a848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbd2e8a8594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbd2e86ec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbd2e878ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbd2881c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbd28811954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbd296756db start_thread at ??:0
@ 0x7fbd2544c61f clone at ??:0
W20260806 06:54:16.813042 20505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:16.813274 20505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:16.813300 20505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:16.814630 20505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:16.814680 20505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:16.814709 20505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:16.816231 20505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:36081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=46593
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:16.816496 20505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:16.816761 20505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:16.822038 20544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:16.822170 20545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:16.822041 20505 server_base.cc:1034] running on GCE node
W20260806 06:54:16.822064 20547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:16.822480 20505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:16.822810 20505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:16.823992 20505 hybrid_clock.cc:648] HybridClock initialized: now 1785999256823978 us; error 28 us; skew 500 ppm
Aug 06 06:54:16 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:16.827018 20505 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:54:16.827520 20505 webserver.cc:460] Webserver started at http://127.3.247.129:46593/ using document root <none> and password file <none>
I20260806 06:54:16.827772 20505 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:16.827843 20505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:16.829347 20505 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:16.830240 20554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:16.830456 20505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:16.830560 20505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9d431fed79572c94f67ae191c1bf8363"
server_key_iv: "5f3db5cca8d0ea0dc57989a3e2347675"
server_key_version: "encryptionkey@0"
I20260806 06:54:16.830947 20505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:16.855158 20505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:16.857738 20505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:16.858001 20505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:16.858307 20505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:16.858806 20561 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:54:16.860970 20505 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:54:16.861071 20505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:54:16.861145 20505 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:54:16.861917 20505 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:54:16.861999 20505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:16.862074 20561 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap starting.
I20260806 06:54:16.869362 20505 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:36081
I20260806 06:54:16.869395 20668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:36081 every 8 connection(s)
I20260806 06:54:16.869849 20505 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:54:16.873528 20561 log.cc:824] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:16.876224 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20505
I20260806 06:54:16.876473 20561 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:54:16.876581 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:46573
--local_ip_for_outbound_sockets=127.3.247.130
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=43805
--webserver_interface=127.3.247.130
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:54:16.876806 20561 tablet_bootstrap.cc:492] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Bootstrap complete.
I20260806 06:54:16.878154 20561 ts_tablet_manager.cc:1397] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260806 06:54:16.881227 20561 raft_consensus.cc:348] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:16.881520 20561 raft_consensus.cc:729] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1a5e6ad9b34adbbb7bad568b7f43ff, State: Initialized, Role: FOLLOWER
I20260806 06:54:16.881698 20561 consensus_queue.cc:260] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:16.882071 20561 ts_tablet_manager.cc:1428] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260806 06:54:16.950021 20671 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:54:16.870579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:55271)
0806 06:54:16.870786 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:16.870832 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:16.871318 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:16.949767 (+ 78449us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":129,"threads_started":1}
W20260806 06:54:16.950335 20669 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:40639 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:40639: Client connection negotiation failed: client connection to 127.3.247.190:40639: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:54:16.950677 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:16.950553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:54035)
0806 06:54:16.950612 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:16.950649 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:54:16.951031 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:16.950944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:54611)
0806 06:54:16.950993 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:16.951015 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:54:16.951156 20669 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:54:16.981878 20673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:16.982095 20673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:16.982121 20673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:16.983433 20673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:16.983479 20673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:16.983507 20673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:16.984999 20673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:46573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=43805
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:16.985255 20673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:16.985512 20673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:16.991194 20682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:16.991163 20679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:16.991189 20680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:16.991588 20673 server_base.cc:1034] running on GCE node
I20260806 06:54:16.991825 20673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:16.992123 20673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:16.993297 20673 hybrid_clock.cc:648] HybridClock initialized: now 1785999256993287 us; error 31 us; skew 500 ppm
Aug 06 06:54:16 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:16.996158 20673 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:54:16.996655 20673 webserver.cc:460] Webserver started at http://127.3.247.130:43805/ using document root <none> and password file <none>
I20260806 06:54:16.996853 20673 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:16.996902 20673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:16.998418 20673 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:16.999043 20689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:16.999275 20673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:54:16.999372 20673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "03006ce657134ddc92e4847d16a5f5a4"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "b7c26823d98dff0c0a8b17bf448077f6"
server_key_iv: "74a398b9a1ecc9d055ccfe90e428c4db"
server_key_version: "encryptionkey@0"
I20260806 06:54:16.999735 20673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:17.010747 20673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:17.012763 20673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:17.012938 20673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:17.013180 20673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:17.013545 20673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:17.013613 20673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:17.013656 20673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:17.013684 20673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:17.019688 20673 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:46573
I20260806 06:54:17.019798 20802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:46573 every 8 connection(s)
I20260806 06:54:17.020140 20673 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:54:17.021224 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.020844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:33711)
0806 06:54:17.021001 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.021075 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260806 06:54:17.021673 20803 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:40639 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:40639: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
I20260806 06:54:17.021992 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.021902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:58141)
0806 06:54:17.021947 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.021974 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 06:54:17.022301 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.022230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:56089)
0806 06:54:17.022266 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.022287 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:54:17.022414 20803 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:54:17.022924 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20673
I20260806 06:54:17.023196 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:33645
--local_ip_for_outbound_sockets=127.3.247.131
--tserver_master_addrs=127.3.247.190:40639
--webserver_port=46513
--webserver_interface=127.3.247.131
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:17.107120 20806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:17.107347 20806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:17.107371 20806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:17.108624 20806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:17.108671 20806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:17.108700 20806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:17.110136 20806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:33645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=46513
--tserver_master_addrs=127.3.247.190:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:17.110325 20806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:17.110508 20806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:54:17.115880 20806 server_base.cc:1034] running on GCE node
W20260806 06:54:17.115794 20814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:17.115789 20812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:17.115794 20811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:17.116338 20806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:17.116616 20806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:17.117846 20806 hybrid_clock.cc:648] HybridClock initialized: now 1785999257117838 us; error 39 us; skew 500 ppm
Aug 06 06:54:17 dist-test-slave-0kls krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999257, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:17.120369 20806 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:54:17.120877 20806 webserver.cc:460] Webserver started at http://127.3.247.131:46513/ using document root <none> and password file <none>
I20260806 06:54:17.121074 20806 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:17.121122 20806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:17.122519 20806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:17.123114 20821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:17.123306 20806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:17.123406 20806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "b1fe05102e9946f78c56119593cf51f9"
format_stamp: "Formatted at 2026-08-06 06:53:35 on dist-test-slave-0kls"
server_key: "9865938a4000026f7c8c37b9f731ad8e"
server_key_iv: "7857e8d0087f55ed488a948b7a3f72b8"
server_key_version: "encryptionkey@0"
I20260806 06:54:17.123765 20806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:17.135694 20806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:17.137996 20806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:17.138146 20806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:17.138437 20806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:17.138772 20806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:17.138827 20806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:17.138870 20806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:17.138897 20806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:17.145319 20806 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:33645
I20260806 06:54:17.145361 20934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:33645 every 8 connection(s)
I20260806 06:54:17.145735 20806 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:17.146900 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.146495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:50515)
0806 06:54:17.146654 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.146739 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260806 06:54:17.147361 20935 heartbeater.cc:644] Failed to heartbeat to 127.3.247.190:40639 (0 consecutive failures): Network error: Failed to ping master at 127.3.247.190:40639: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
I20260806 06:54:17.147696 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.147588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:39831)
0806 06:54:17.147642 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.147674 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:54:17.147970 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.147914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:44081)
0806 06:54:17.147935 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.147957 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 06:54:17.148119 20935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:54:17.148486 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 20806
I20260806 06:54:17.951879 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:17.951578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:54533)
0806 06:54:17.951741 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:17.951803 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260806 06:54:18.022974 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.022768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:44491)
0806 06:54:18.022883 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:18.022923 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260806 06:54:18.148727 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.148491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:54765)
0806 06:54:18.148630 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:18.148669 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":58,"threads_started":1}
I20260806 06:54:18.575271 20942 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:18.575397 20942 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:18.575695 20942 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573), b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
Aug 06 06:54:18 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.130@KRBTEST.COM
Aug 06 06:54:18 dist-test-slave-0kls krb5kdc[19368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999256, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.131@KRBTEST.COM
I20260806 06:54:18.586972 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.575899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:46573 (local address 127.3.247.129:58387)
0806 06:54:18.576086 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:18.576097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:18.576154 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:18.577145 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:18.577148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:18.577533 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:18.577733 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:18.577740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.578962 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.578963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:18.579713 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:18.579715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.580304 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.580305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:18.580409 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:18.581069 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:18.581106 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:18.582820 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:18.585116 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585123 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:18.585125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:18.585440 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:18.585696 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585696 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:18.585696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:18.585881 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:18.586400 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:18.586405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:18.586568 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 06:54:18.586677 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:54:18.586858 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":68,"thread_start_us":84,"threads_started":1}
I20260806 06:54:18.587034 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.576076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:41201 (local address 127.3.247.131:33645)
0806 06:54:18.576238 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:18.576258 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:18.576285 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:54:18.576896 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:18.576899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:18.576943 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:18.577094 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:18.577879 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.578810 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.579804 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.580205 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.580327 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:18.583361 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:18.583364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:18.583365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:18.583385 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:18.585239 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585667 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:18.585670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:18.585670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:18.585822 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:18.586028 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:18.586029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:18.586030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:18.586318 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:18.586406 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:18.586805 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 06:54:18.586929 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":52,"threads_started":1}
I20260806 06:54:18.587285 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.575898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:33645 (local address 127.3.247.129:41201)
0806 06:54:18.576073 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:18.576088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:18.576158 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:18.577145 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:18.577148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:18.577534 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:18.577733 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:18.577740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.578915 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.578916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:18.579712 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:18.579714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.580297 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.580299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:18.580408 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:18.581088 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:18.581097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:18.583260 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:18.585314 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:18.585315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:18.585567 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:18.585889 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:18.585890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:18.585964 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:18.586397 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:18.586405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:18.586582 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 06:54:18.586694 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:54:18.587213 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":393,"thread_start_us":83,"threads_started":1}
I20260806 06:54:18.587302 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.576042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:58387 (local address 127.3.247.130:46573)
0806 06:54:18.576260 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:18.576268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:18.576285 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:54:18.576919 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:18.576922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:18.576966 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:18.577098 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:18.577899 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.578874 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.579815 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:18.580217 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:18.580337 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:18.582924 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:18.582928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:18.582928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:18.582938 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:18.585005 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585519 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:18.585521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:18.585522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:18.585627 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:18.585963 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:18.585963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:18.585964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:18.586318 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:18.586407 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:18.586777 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 06:54:18.586939 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":60,"threads_started":1}
W20260806 06:54:18.588033 20558 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
W20260806 06:54:18.588186 20557 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:54:18.588240 20557 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac1a5e6ad9b34adbbb7bad568b7f43ff; no voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9
I20260806 06:54:18.588339 20942 raft_consensus.cc:2726] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:54:18.952561 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:18.952400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:49425)
0806 06:54:18.952482 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:18.952521 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 06:54:19.023674 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:19.023436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:52655)
0806 06:54:19.023565 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:19.023619 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260806 06:54:19.149606 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:19.149311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:47197)
0806 06:54:19.149482 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:19.149519 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260806 06:54:19.953341 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:19.953085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:60175)
0806 06:54:19.953244 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:19.953294 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260806 06:54:20.024459 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:20.024222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:33249)
0806 06:54:20.024368 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:20.024412 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260806 06:54:20.150285 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:20.150068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:48987)
0806 06:54:20.150201 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:20.150244 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260806 06:54:20.480312 20952 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:20.480409 20952 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:20.480551 20952 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573), b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
W20260806 06:54:20.481011 20558 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
W20260806 06:54:20.481168 20557 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:54:20.481249 20557 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac1a5e6ad9b34adbbb7bad568b7f43ff; no voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9
I20260806 06:54:20.481348 20952 raft_consensus.cc:2726] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:54:20.954126 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:20.953877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:49729)
0806 06:54:20.954004 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:20.954038 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260806 06:54:21.025223 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:21.024986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:57317)
0806 06:54:21.025133 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:21.025178 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260806 06:54:21.151007 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:21.150780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:55409)
0806 06:54:21.150920 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:21.150964 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260806 06:54:21.954872 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:21.954662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:48071)
0806 06:54:21.954788 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:21.954825 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260806 06:54:22.026033 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:22.025712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:39883)
0806 06:54:22.025902 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:22.025972 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":60,"threads_started":1}
I20260806 06:54:22.151705 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:22.151478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:57839)
0806 06:54:22.151625 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:22.151658 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260806 06:54:22.726253 20959 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:22.726356 20959 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:22.726542 20959 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573), b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
W20260806 06:54:22.726961 20557 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
W20260806 06:54:22.727061 20558 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:54:22.727166 20558 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac1a5e6ad9b34adbbb7bad568b7f43ff; no voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9
I20260806 06:54:22.727252 20959 raft_consensus.cc:2726] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:54:22.955761 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:22.955485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:51737)
0806 06:54:22.955658 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:22.955712 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260806 06:54:23.026852 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:23.026554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:59651)
0806 06:54:23.026732 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:23.026798 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260806 06:54:23.152549 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:23.152255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:50735)
0806 06:54:23.152462 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:23.152503 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":143,"threads_started":1}
I20260806 06:54:23.956561 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:23.956317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:34977)
0806 06:54:23.956463 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:23.956510 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260806 06:54:24.027691 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:24.027409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:53635)
0806 06:54:24.027601 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:24.027643 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260806 06:54:24.153625 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:24.153115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:44871)
0806 06:54:24.153306 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:24.153553 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260806 06:54:24.957496 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:24.957210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:45039)
0806 06:54:24.957386 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:24.957437 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260806 06:54:25.028543 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:25.028291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:38499)
0806 06:54:25.028452 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:25.028496 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260806 06:54:25.154579 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:25.154333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:54903)
0806 06:54:25.154485 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:25.154533 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260806 06:54:25.958395 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:25.958145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:49561)
0806 06:54:25.958299 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:25.958346 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260806 06:54:26.029479 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:26.029159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:46985)
0806 06:54:26.029304 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:26.029397 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260806 06:54:26.155514 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:26.155227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:53575)
0806 06:54:26.155409 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:26.155456 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260806 06:54:26.600987 20972 raft_consensus.cc:482] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:26.601118 20972 raft_consensus.cc:504] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03006ce657134ddc92e4847d16a5f5a4" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 46573 } } peers { permanent_uuid: "b1fe05102e9946f78c56119593cf51f9" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 33645 } } peers { permanent_uuid: "ac1a5e6ad9b34adbbb7bad568b7f43ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 36081 } }
I20260806 06:54:26.601325 20972 leader_election.cc:288] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573), b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645)
W20260806 06:54:26.601824 20558 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b1fe05102e9946f78c56119593cf51f9 (127.3.247.131:33645): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
W20260806 06:54:26.601907 20557 leader_election.cc:341] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03006ce657134ddc92e4847d16a5f5a4 (127.3.247.130:46573): Not found: Tablet not found: f6eb6674b0fa44eabdf711e9aaad0aef
I20260806 06:54:26.602005 20557 leader_election.cc:302] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac1a5e6ad9b34adbbb7bad568b7f43ff; no voters: 03006ce657134ddc92e4847d16a5f5a4, b1fe05102e9946f78c56119593cf51f9
I20260806 06:54:26.602092 20972 raft_consensus.cc:2726] T f6eb6674b0fa44eabdf711e9aaad0aef P ac1a5e6ad9b34adbbb7bad568b7f43ff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:54:26.959286 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:26.959026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:45315)
0806 06:54:26.959194 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:26.959239 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260806 06:54:27.030297 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:27.030065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:34607)
0806 06:54:27.030195 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:27.030250 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260806 06:54:27.156427 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:27.156161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:60247)
0806 06:54:27.156332 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:27.156379 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260806 06:54:27.960089 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:27.959834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:36657)
0806 06:54:27.959995 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:27.960042 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260806 06:54:28.031172 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:28.030923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:42867)
0806 06:54:28.031078 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:28.031125 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260806 06:54:28.157279 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:28.157030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:39287)
0806 06:54:28.157178 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:28.157228 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 06:54:28.960994 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:28.960749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:46469)
0806 06:54:28.960895 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:28.960943 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260806 06:54:29.032083 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:29.031778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:37329)
0806 06:54:29.031992 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:29.032034 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
I20260806 06:54:29.158252 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:29.157959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:53605)
0806 06:54:29.158129 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:29.158175 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260806 06:54:29.961879 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:29.961568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:35723)
0806 06:54:29.961717 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:29.961764 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260806 06:54:30.032804 20983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:30.032584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:34201)
0806 06:54:30.032708 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:30.032748 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260806 06:54:30.159008 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:30.158786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:54597)
0806 06:54:30.158924 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:30.158962 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260806 06:54:30.962687 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:30.962437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:50079)
0806 06:54:30.962588 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:30.962636 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260806 06:54:31.033605 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:31.033361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:47409)
0806 06:54:31.033513 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:31.033558 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260806 06:54:31.159809 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:31.159544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:41687)
0806 06:54:31.159702 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:31.159754 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":61,"threads_started":1}
I20260806 06:54:31.963461 20988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:31.963256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.129:33899)
0806 06:54:31.963378 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:31.963416 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260806 06:54:32.034437 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:32.034199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.130:34529)
0806 06:54:32.034348 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:32.034390 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260806 06:54:32.151571 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 20505
I20260806 06:54:32.156744 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 20673
I20260806 06:54:32.160670 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:32.160429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:40639 (local address 127.3.247.131:60553)
0806 06:54:32.160557 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:32.160602 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.190:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":53,"threads_started":1}
I20260806 06:54:32.161199 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 20806
2026-08-06T06:54:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58428 ms)
[----------] 30 tests from SecurityITest (231800 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:32 dist-test-slave-0kls krb5kdc[20993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:32 dist-test-slave-0kls krb5kdc[20993](info): set up 2 sockets
Aug 06 06:54:32 dist-test-slave-0kls krb5kdc[20993](info): commencing operation
krb5kdc: starting...
W20260806 06:54:33.205660 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:54:33.329349 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38039
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:33.410859 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:33.411130 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:33.411187 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:33.412770 21006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:33.412832 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:33.412853 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:33.412876 21006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:33.412895 21006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:33.415572 21006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:33.415904 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:33.416190 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:33.421563 21011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.421569 21012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.421826 21006 server_base.cc:1034] running on GCE node
W20260806 06:54:33.421979 21014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.422252 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:33.422600 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:33.423859 21006 hybrid_clock.cc:648] HybridClock initialized: now 1785999273423834 us; error 44 us; skew 500 ppm
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:33.426581 21006 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:33.427126 21006 webserver.cc:460] Webserver started at http://127.3.247.190:33787/ using document root <none> and password file <none>
I20260806 06:54:33.427398 21006 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:33.427470 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:33.427623 21006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:33.428721 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "0d0289e528ee2eee9fcd8a8e58cd4c5f"
server_key_iv: "adda23e51181ef5aaf96fd1b5c44ab64"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.429090 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "0d0289e528ee2eee9fcd8a8e58cd4c5f"
server_key_iv: "adda23e51181ef5aaf96fd1b5c44ab64"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.430521 21006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:33.431288 21021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.431457 21006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:33.431558 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "0d0289e528ee2eee9fcd8a8e58cd4c5f"
server_key_iv: "adda23e51181ef5aaf96fd1b5c44ab64"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.431633 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:33.451983 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:33.454465 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:33.454656 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:33.458617 21006 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38039
I20260806 06:54:33.458657 21073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38039 every 8 connection(s)
I20260806 06:54:33.459028 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:33.459918 21074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:33.462252 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: Bootstrap starting.
I20260806 06:54:33.462832 21074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:33.463119 21074 log.cc:824] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:33.463835 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: No bootstrap required, opened a new log
I20260806 06:54:33.464500 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21006
I20260806 06:54:33.464632 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:33.464847 4062 external_mini_cluster.cc:1428] Setting key 2728a3cf02c404c4b5e7a0a472e76675
I20260806 06:54:33.466065 21074 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } }
I20260806 06:54:33.466244 21074 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:33.466284 21074 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbb39d3e1a1c493cafbce0bad6b9d9ab, State: Initialized, Role: FOLLOWER
I20260806 06:54:33.466444 21074 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } }
I20260806 06:54:33.466547 21074 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:33.466591 21074 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:33.466639 21074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:33.467774 21074 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } }
I20260806 06:54:33.467883 21074 leader_election.cc:302] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbb39d3e1a1c493cafbce0bad6b9d9ab; no voters:
I20260806 06:54:33.468037 21074 leader_election.cc:288] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:33.468096 21079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:33.468274 21074 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:33.468319 21079 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [term 1 LEADER]: Becoming Leader. State: Replica: cbb39d3e1a1c493cafbce0bad6b9d9ab, State: Running, Role: LEADER
I20260806 06:54:33.468503 21079 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } }
I20260806 06:54:33.468981 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } } }
I20260806 06:54:33.469076 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:33.469321 21081 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbb39d3e1a1c493cafbce0bad6b9d9ab. Latest consensus state: current_term: 1 leader_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbb39d3e1a1c493cafbce0bad6b9d9ab" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38039 } } }
I20260806 06:54:33.469446 21081 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:33.469632 21088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:33.470193 21088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:33.472877 21088 catalog_manager.cc:1347] Generated new cluster ID: 12f9611f0fee462a809dca0a9266ee9a
I20260806 06:54:33.472927 21088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:33.477072 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.465404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.3.247.190:38039)
0806 06:54:33.465624 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.465633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.465670 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:54:33.466018 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.466028 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.466093 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.466256 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:33.467048 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.468526 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.469733 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.470221 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.470347 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.473263 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:33.473267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:33.473267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:33.473281 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:33.475330 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.475726 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.475729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.475730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.475866 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.476071 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.476073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.476073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.476396 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:33.476546 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.476656 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.476832 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":104,"threads_started":1}
I20260806 06:54:33.484714 21088 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:33.485248 21088 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:33.494861 21088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: Generated new TSK 0
I20260806 06:54:33.495074 21088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:54:33.524417 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:33.607661 21100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:33.607868 21100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:33.607898 21100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:33.609067 21100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:33.609107 21100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:33.609133 21100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:33.610539 21100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:33.610733 21100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:33.610901 21100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:33.616003 21106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.615994 21108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.616027 21105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.616044 21100 server_base.cc:1034] running on GCE node
I20260806 06:54:33.616402 21100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:33.616654 21100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:33.617852 21100 hybrid_clock.cc:648] HybridClock initialized: now 1785999273617833 us; error 29 us; skew 500 ppm
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:33.620277 21100 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:54:33.620726 21100 webserver.cc:460] Webserver started at http://127.3.247.129:37191/ using document root <none> and password file <none>
I20260806 06:54:33.620960 21100 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:33.621026 21100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:33.621178 21100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:33.622316 21100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "97162fd780474258aaadc9eb171b107a"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "2776f9dbd97b005b0e67f9cf50d181cc"
server_key_iv: "b72afd566a81ab88ecb27fa7f7acca0c"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.622673 21100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "97162fd780474258aaadc9eb171b107a"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "2776f9dbd97b005b0e67f9cf50d181cc"
server_key_iv: "b72afd566a81ab88ecb27fa7f7acca0c"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.623977 21100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:33.624716 21115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.624884 21100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:33.624986 21100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "97162fd780474258aaadc9eb171b107a"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "2776f9dbd97b005b0e67f9cf50d181cc"
server_key_iv: "b72afd566a81ab88ecb27fa7f7acca0c"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.625058 21100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:33.635836 21100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:33.637609 21100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:33.637773 21100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:33.638077 21100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:33.638398 21100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:33.638455 21100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.638489 21100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:33.638515 21100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.644083 21100 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:46623
I20260806 06:54:33.644158 21228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:46623 every 8 connection(s)
I20260806 06:54:33.644484 21100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:54:33.649572 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21100
I20260806 06:54:33.649703 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:33.649951 4062 external_mini_cluster.cc:1428] Setting key 0d5cd3f1f3512a71244dd3e57afbabe6
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:33.654460 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.645220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:54551 (local address 127.3.247.190:38039)
0806 06:54:33.645280 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.645282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.645453 (+ 171us) server_negotiation.cc:407] Connection header received
0806 06:54:33.646045 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.646047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.646065 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.646128 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:33.647091 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.647606 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.648614 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.648824 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.648898 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.651211 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:33.651213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:33.651213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:33.651221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:33.652902 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.653430 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.653431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.653432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.653483 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.653761 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.653762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.653762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.653922 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:33.653977 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.654323 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.654393 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:33.654774 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.645184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38039 (local address 127.3.247.129:54551)
0806 06:54:33.645322 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.645368 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.645944 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.646205 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.646209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.646798 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:33.646992 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.646997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.647709 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.647710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.648516 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.648518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.648918 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.648919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.649047 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.649554 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:33.649565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:33.651116 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:33.652991 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.653013 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.653018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.653350 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.653549 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.653550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.653550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.653696 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.653992 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:33.653998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:33.654172 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.654297 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.654486 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260806 06:54:33.655418 21229 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38039
I20260806 06:54:33.655525 21229 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:33.655701 21229 heartbeater.cc:511] Master 127.3.247.190:38039 requested a full tablet report, sending...
I20260806 06:54:33.656239 21038 ts_manager.cc:194] Registered new tserver with Master: 97162fd780474258aaadc9eb171b107a (127.3.247.129:46623)
I20260806 06:54:33.656868 21038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:54551
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:54:33.676847 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:33.759459 21234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:33.759732 21234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:33.759778 21234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:33.760974 21234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:33.761050 21234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:33.761099 21234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:33.762686 21234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:33.762986 21234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:33.763268 21234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:33.768301 21240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.768319 21242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.768328 21239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.768592 21234 server_base.cc:1034] running on GCE node
I20260806 06:54:33.768918 21234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:33.769192 21234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:33.770357 21234 hybrid_clock.cc:648] HybridClock initialized: now 1785999273770352 us; error 37 us; skew 500 ppm
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:33.772955 21234 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:54:33.773443 21234 webserver.cc:460] Webserver started at http://127.3.247.130:42779/ using document root <none> and password file <none>
I20260806 06:54:33.773685 21234 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:33.773762 21234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:33.773937 21234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:33.774987 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "cc733ded9e3f49b9bf19d2e896749c09"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "8de4a031abf03ee3df1b9648a564b16d"
server_key_iv: "2e1fc5828715ff572b1f5c08383ea61a"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.775341 21234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "cc733ded9e3f49b9bf19d2e896749c09"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "8de4a031abf03ee3df1b9648a564b16d"
server_key_iv: "2e1fc5828715ff572b1f5c08383ea61a"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.776638 21234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:33.777328 21249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.777500 21234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:33.777602 21234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "cc733ded9e3f49b9bf19d2e896749c09"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "8de4a031abf03ee3df1b9648a564b16d"
server_key_iv: "2e1fc5828715ff572b1f5c08383ea61a"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.777683 21234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:33.786270 21234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:33.788565 21234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:33.788733 21234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:33.788964 21234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:33.789279 21234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:33.789335 21234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.789377 21234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:33.789404 21234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.795363 21234 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:35191
I20260806 06:54:33.795423 21362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:35191 every 8 connection(s)
I20260806 06:54:33.795774 21234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:33.801769 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21234
I20260806 06:54:33.801901 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:33.802131 4062 external_mini_cluster.cc:1428] Setting key a7ce8a1b81da14c9f531bc628f4e9b47
I20260806 06:54:33.805230 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.796601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:51797 (local address 127.3.247.190:38039)
0806 06:54:33.796674 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.796677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.796898 (+ 221us) server_negotiation.cc:407] Connection header received
0806 06:54:33.797365 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.797366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.797376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.797428 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:33.798408 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.798898 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.799835 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.800013 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.800067 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.802269 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:33.802270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:33.802270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:33.802277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:33.803767 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.804193 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.804194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.804195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.804260 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.804540 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.804541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.804541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.804659 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:33.804732 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.805078 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.805158 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:54:33.805481 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.796580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38039 (local address 127.3.247.130:51797)
0806 06:54:33.796778 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.796814 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.797279 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.797479 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.797482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.798094 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:33.798295 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.798300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.798980 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.798981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.799759 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.799760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.800086 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.800087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.800188 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.800746 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:33.800754 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:33.802177 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:33.803868 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.803875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.803877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.804103 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.804340 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.804341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.804341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.804462 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.804736 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:33.804743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:33.804916 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.805030 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.805204 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":116,"threads_started":1}
I20260806 06:54:33.806038 21363 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38039
I20260806 06:54:33.806133 21363 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:33.806331 21363 heartbeater.cc:511] Master 127.3.247.190:38039 requested a full tablet report, sending...
I20260806 06:54:33.806775 21038 ts_manager.cc:194] Registered new tserver with Master: cc733ded9e3f49b9bf19d2e896749c09 (127.3.247.130:35191)
I20260806 06:54:33.807149 21038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:51797
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:54:33.828346 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:33.909705 21368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:33.910022 21368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:33.910063 21368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:33.911216 21368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:33.911294 21368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:33.911337 21368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:33.912796 21368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:38039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:33.913069 21368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:33.913357 21368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:33.918318 21376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:33.918313 21374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.918457 21368 server_base.cc:1034] running on GCE node
W20260806 06:54:33.918344 21373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:33.918809 21368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:33.919078 21368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:33.920262 21368 hybrid_clock.cc:648] HybridClock initialized: now 1785999273920257 us; error 35 us; skew 500 ppm
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:33.922839 21368 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:54:33.923264 21368 webserver.cc:460] Webserver started at http://127.3.247.131:45353/ using document root <none> and password file <none>
I20260806 06:54:33.923494 21368 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:33.923566 21368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:33.923702 21368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:33.924739 21368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "e32f85c5aad5426cb82953dd09153186"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "318d4f922272bb746fe97fe19a428919"
server_key_iv: "0aa40c28c1dae68852f98b168182d514"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.925102 21368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "e32f85c5aad5426cb82953dd09153186"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "318d4f922272bb746fe97fe19a428919"
server_key_iv: "0aa40c28c1dae68852f98b168182d514"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.926492 21368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:33.927184 21383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.927369 21368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.927464 21368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "e32f85c5aad5426cb82953dd09153186"
format_stamp: "Formatted at 2026-08-06 06:54:33 on dist-test-slave-0kls"
server_key: "318d4f922272bb746fe97fe19a428919"
server_key_iv: "0aa40c28c1dae68852f98b168182d514"
server_key_version: "encryptionkey@0"
I20260806 06:54:33.927551 21368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:33.938323 21368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:33.940263 21368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:33.940428 21368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:33.940659 21368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:33.940973 21368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:33.941028 21368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.941071 21368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:33.941098 21368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:33.947033 21368 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:44619
I20260806 06:54:33.947084 21496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:44619 every 8 connection(s)
I20260806 06:54:33.947438 21368 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:33.953290 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21368
I20260806 06:54:33.953423 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:33.953647 4062 external_mini_cluster.cc:1428] Setting key 1ba765b80858915e45c355cbb068a333
Aug 06 06:54:33 dist-test-slave-0kls krb5kdc[20993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999273, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:33.957299 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.948198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:37153 (local address 127.3.247.190:38039)
0806 06:54:33.948270 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.948272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.948447 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:54:33.949278 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.949280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.949293 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.949367 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:33.950449 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.950961 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.951982 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.952191 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.952282 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.954762 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:33.954764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:33.954764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:33.954771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:33.955935 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.956323 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.956325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.956325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.956392 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.956646 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.956646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.956647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.956770 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:33.956850 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.957126 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.957206 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:54:33.957504 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.948168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:38039 (local address 127.3.247.131:37153)
0806 06:54:33.948322 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.948363 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.949092 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.949443 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.949447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.950155 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:33.950350 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.950354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.951077 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.951078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.951876 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.951878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.952292 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.952293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.952431 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.952878 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:33.952886 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:33.954646 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:33.956011 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.956018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.956021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.956244 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.956458 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:33.956459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:33.956459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:33.956572 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:33.956840 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:33.956846 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:33.956985 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.957095 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.957260 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260806 06:54:33.957991 21497 heartbeater.cc:347] Connected to a master server at 127.3.247.190:38039
I20260806 06:54:33.958108 21497 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:33.958318 21497 heartbeater.cc:511] Master 127.3.247.190:38039 requested a full tablet report, sending...
I20260806 06:54:33.958688 21038 ts_manager.cc:194] Registered new tserver with Master: e32f85c5aad5426cb82953dd09153186 (127.3.247.131:44619)
I20260806 06:54:33.959036 21038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:37153
I20260806 06:54:33.965430 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:33.973062 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.966800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.3.247.190:38039)
0806 06:54:33.966861 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.966864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.966975 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:54:33.967064 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.967065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.967075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.967131 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:33.967731 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.968308 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.969066 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.969289 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.969368 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.970706 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:33.970707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:33.970707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:33.970715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:33.971882 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.972228 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.972229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.972230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.972293 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:33.972494 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:33.972495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:33.972495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:33.972647 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:33.972762 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.972876 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.972956 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:54:33.974638 21038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:33.976269 21038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:33.982770 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.3.247.130:35191)
0806 06:54:33.978789 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.978794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.978833 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:54:33.978910 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.978912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.978928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.979026 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:33.979220 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980404 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.981700 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.982330 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.982406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.982466 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.982628 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.982674 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260806 06:54:33.982764 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:35191 (local address 127.0.0.1:54476)
0806 06:54:33.978781 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.978790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.978841 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.979049 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.979052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.979054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:33.979132 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.979134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980497 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.980497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.981613 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.981616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.982416 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.982416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.982504 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.982519 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.982605 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.982665 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":61,"thread_start_us":60,"threads_started":1}
I20260806 06:54:33.982909 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:46623 (local address 127.0.0.1:52602)
0806 06:54:33.978565 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.978603 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.978668 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.979141 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.979142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.979144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:33.979192 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.979194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980706 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.980707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.981830 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.981832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.982715 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.982716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.982771 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.982775 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.982806 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.982849 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"spinlock_wait_cycles":384,"thread_start_us":64,"threads_started":1}
I20260806 06:54:33.983268 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.3.247.129:46623)
0806 06:54:33.978979 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.978982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.979009 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:54:33.979047 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.979049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.979064 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.979145 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:33.979314 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980404 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.981921 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.982628 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.983034 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.983097 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.983129 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.983181 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260806 06:54:33.983753 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:44619 (local address 127.0.0.1:49864)
0806 06:54:33.978990 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:33.979001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:33.979054 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:33.979398 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:33.979399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:33.979401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:33.979443 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.979445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980433 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.980434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.982120 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:33.982122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.983527 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.983528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:33.983588 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.983592 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:33.983649 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:54:33.983686 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":47,"thread_start_us":54,"threads_started":1}
I20260806 06:54:33.983664 21297 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 (DEFAULT_TABLE table=test-table [id=689360bef5394012ac524fd7dd6672c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:33.983803 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:33.978635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.3.247.131:44619)
0806 06:54:33.978851 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:33.978855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:33.979110 (+ 255us) server_negotiation.cc:407] Connection header received
0806 06:54:33.979153 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:33.979155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:33.979169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:33.979315 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:33.979518 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.980334 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.982200 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:33.983370 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:33.983439 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:33.983490 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:33.983684 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 06:54:33.983729 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":101,"threads_started":1}
I20260806 06:54:33.983932 21163 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 (DEFAULT_TABLE table=test-table [id=689360bef5394012ac524fd7dd6672c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:33.984054 21297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4bb1e96d044d4e9e323f15f74e0149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:33.984262 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4bb1e96d044d4e9e323f15f74e0149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:33.984431 21431 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 (DEFAULT_TABLE table=test-table [id=689360bef5394012ac524fd7dd6672c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:33.984699 21431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4bb1e96d044d4e9e323f15f74e0149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:33.986744 21511 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Bootstrap starting.
I20260806 06:54:33.987457 21511 tablet_bootstrap.cc:654] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:33.987803 21512 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Bootstrap starting.
I20260806 06:54:33.987969 21513 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Bootstrap starting.
I20260806 06:54:33.988298 21511 log.cc:824] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:33.988509 21512 tablet_bootstrap.cc:654] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:33.988605 21513 tablet_bootstrap.cc:654] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:33.989117 21513 log.cc:824] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:33.989140 21511 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: No bootstrap required, opened a new log
I20260806 06:54:33.989148 21512 log.cc:824] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:33.989199 21511 ts_tablet_manager.cc:1397] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:54:33.989831 21512 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: No bootstrap required, opened a new log
I20260806 06:54:33.989864 21513 tablet_bootstrap.cc:492] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: No bootstrap required, opened a new log
I20260806 06:54:33.989898 21512 ts_tablet_manager.cc:1397] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:33.989912 21513 ts_tablet_manager.cc:1397] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:33.990839 21511 raft_consensus.cc:348] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.990962 21511 raft_consensus.cc:374] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:33.990984 21511 raft_consensus.cc:729] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc733ded9e3f49b9bf19d2e896749c09, State: Initialized, Role: FOLLOWER
I20260806 06:54:33.991101 21511 consensus_queue.cc:260] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.991380 21511 ts_tablet_manager.cc:1428] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:33.991389 21363 heartbeater.cc:503] Master 127.3.247.190:38039 was elected leader, sending a full tablet report...
I20260806 06:54:33.991806 21513 raft_consensus.cc:348] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.991806 21512 raft_consensus.cc:348] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.991964 21513 raft_consensus.cc:374] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:33.991968 21512 raft_consensus.cc:374] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:33.992000 21512 raft_consensus.cc:729] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97162fd780474258aaadc9eb171b107a, State: Initialized, Role: FOLLOWER
I20260806 06:54:33.992000 21513 raft_consensus.cc:729] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32f85c5aad5426cb82953dd09153186, State: Initialized, Role: FOLLOWER
I20260806 06:54:33.992103 21513 consensus_queue.cc:260] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.992103 21512 consensus_queue.cc:260] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:33.992362 21512 ts_tablet_manager.cc:1428] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:33.992363 21513 ts_tablet_manager.cc:1428] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:33.992542 21497 heartbeater.cc:503] Master 127.3.247.190:38039 was elected leader, sending a full tablet report...
I20260806 06:54:33.992599 21229 heartbeater.cc:503] Master 127.3.247.190:38039 was elected leader, sending a full tablet report...
W20260806 06:54:34.046355 21364 tablet.cc:2369] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:54:34.145187 21230 tablet.cc:2369] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:54:34.198099 21498 tablet.cc:2369] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:34.264974 21518 raft_consensus.cc:482] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:34.265128 21518 raft_consensus.cc:504] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:34.265518 21518 leader_election.cc:288] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97162fd780474258aaadc9eb171b107a (127.3.247.129:46623), cc733ded9e3f49b9bf19d2e896749c09 (127.3.247.130:35191)
I20260806 06:54:34.268502 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.265732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:46623 (local address 127.3.247.131:43427)
0806 06:54:34.265877 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.265890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.265953 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.266156 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.266157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.266158 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:34.266230 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.266231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.266880 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.266881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.267713 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.267716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.268243 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.268244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.268325 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.268328 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.268391 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.268430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 06:54:34.268503 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.265805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:43427 (local address 127.3.247.129:46623)
0806 06:54:34.265864 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.265867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.266053 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:54:34.266085 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.266086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.266097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.266153 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:34.266300 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.266786 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.267799 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.268152 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.268236 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.268245 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.268396 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.268430 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:54:34.268631 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.265746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:35191 (local address 127.3.247.131:32893)
0806 06:54:34.265964 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.265975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.266048 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.266215 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.266216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.266218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:34.266289 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.266291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.266945 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.266946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.267793 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.267796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.268350 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.268351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.268425 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.268429 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.268508 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.268549 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":157,"threads_started":1}
I20260806 06:54:34.268620 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.265879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:32893 (local address 127.3.247.130:35191)
0806 06:54:34.265933 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.265935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.266025 (+ 90us) server_negotiation.cc:407] Connection header received
0806 06:54:34.266117 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.266118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.266126 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.266211 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:34.266375 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.266860 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.267878 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.268253 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.268349 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.268357 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.268515 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.268551 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:54:34.268981 21317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4bb1e96d044d4e9e323f15f74e0149" candidate_uuid: "e32f85c5aad5426cb82953dd09153186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc733ded9e3f49b9bf19d2e896749c09" is_pre_election: true
I20260806 06:54:34.269042 21183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4bb1e96d044d4e9e323f15f74e0149" candidate_uuid: "e32f85c5aad5426cb82953dd09153186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97162fd780474258aaadc9eb171b107a" is_pre_election: true
I20260806 06:54:34.269135 21317 raft_consensus.cc:2454] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32f85c5aad5426cb82953dd09153186 in term 0.
I20260806 06:54:34.269234 21183 raft_consensus.cc:2454] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32f85c5aad5426cb82953dd09153186 in term 0.
I20260806 06:54:34.269373 21385 leader_election.cc:302] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc733ded9e3f49b9bf19d2e896749c09, e32f85c5aad5426cb82953dd09153186; no voters:
I20260806 06:54:34.269507 21518 raft_consensus.cc:2781] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:34.269590 21518 raft_consensus.cc:482] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:34.269626 21518 raft_consensus.cc:3037] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:34.270500 21518 raft_consensus.cc:504] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:34.270668 21518 leader_election.cc:288] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [CANDIDATE]: Term 1 election: Requested vote from peers 97162fd780474258aaadc9eb171b107a (127.3.247.129:46623), cc733ded9e3f49b9bf19d2e896749c09 (127.3.247.130:35191)
I20260806 06:54:34.270784 21183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4bb1e96d044d4e9e323f15f74e0149" candidate_uuid: "e32f85c5aad5426cb82953dd09153186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97162fd780474258aaadc9eb171b107a"
I20260806 06:54:34.270838 21317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4bb1e96d044d4e9e323f15f74e0149" candidate_uuid: "e32f85c5aad5426cb82953dd09153186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc733ded9e3f49b9bf19d2e896749c09"
I20260806 06:54:34.270888 21183 raft_consensus.cc:3037] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:34.270947 21317 raft_consensus.cc:3037] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:34.271595 21183 raft_consensus.cc:2454] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32f85c5aad5426cb82953dd09153186 in term 1.
I20260806 06:54:34.271662 21317 raft_consensus.cc:2454] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32f85c5aad5426cb82953dd09153186 in term 1.
I20260806 06:54:34.271844 21385 leader_election.cc:302] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc733ded9e3f49b9bf19d2e896749c09, e32f85c5aad5426cb82953dd09153186; no voters:
I20260806 06:54:34.271986 21518 raft_consensus.cc:2781] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:34.272159 21518 raft_consensus.cc:686] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 1 LEADER]: Becoming Leader. State: Replica: e32f85c5aad5426cb82953dd09153186, State: Running, Role: LEADER
I20260806 06:54:34.272300 21518 consensus_queue.cc:237] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } }
I20260806 06:54:34.273101 21037 catalog_manager.cc:5495] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 reported cstate change: term changed from 0 to 1, leader changed from <none> to e32f85c5aad5426cb82953dd09153186 (127.3.247.131). New cstate: current_term: 1 leader_uuid: "e32f85c5aad5426cb82953dd09153186" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e32f85c5aad5426cb82953dd09153186" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 44619 } health_report { overall_health: HEALTHY } } }
I20260806 06:54:34.338325 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.336039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.3.247.131:44619)
0806 06:54:34.336108 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.336110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.336140 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:54:34.336176 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.336177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.336200 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.336276 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:34.336411 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.336870 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.337585 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.337766 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.337839 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.337976 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:34.338076 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:34.338135 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.338163 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.338180 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:54:34.340085 21183 raft_consensus.cc:1261] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 1 FOLLOWER]: Refusing update from remote peer e32f85c5aad5426cb82953dd09153186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:34.340090 21317 raft_consensus.cc:1261] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 1 FOLLOWER]: Refusing update from remote peer e32f85c5aad5426cb82953dd09153186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:34.340507 21521 consensus_queue.cc:1035] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97162fd780474258aaadc9eb171b107a" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 46623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:34.340649 21518 consensus_queue.cc:1035] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc733ded9e3f49b9bf19d2e896749c09" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 35191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:34.342839 21524 mvcc.cc:205] Tried to move back new op lower bound from 7315453027694419968 to 7315453027419897856. Current Snapshot: MvccSnapshot[applied={T|T < 7315453027694419968}]
I20260806 06:54:34.343065 21525 mvcc.cc:205] Tried to move back new op lower bound from 7315453027694419968 to 7315453027419897856. Current Snapshot: MvccSnapshot[applied={T|T < 7315453027694419968}]
I20260806 06:54:34.343156 21526 mvcc.cc:205] Tried to move back new op lower bound from 7315453027694419968 to 7315453027419897856. Current Snapshot: MvccSnapshot[applied={T|T < 7315453027694419968}]
I20260806 06:54:34.345546 21038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:34.345624 21038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:34.346395 21038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbb39d3e1a1c493cafbce0bad6b9d9ab: Sending DeleteTablet for 3 replicas of tablet 5b4bb1e96d044d4e9e323f15f74e0149
I20260806 06:54:34.346727 21431 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:34 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:49864
I20260806 06:54:34.346745 21297 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:34 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:54476
I20260806 06:54:34.346946 21163 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4bb1e96d044d4e9e323f15f74e0149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:34 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:52602
I20260806 06:54:34.347019 21534 tablet_replica.cc:330] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: stopping tablet replica
I20260806 06:54:34.347126 21534 raft_consensus.cc:2229] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:34.347226 21536 tablet_replica.cc:330] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a: stopping tablet replica
I20260806 06:54:34.347245 21534 raft_consensus.cc:2258] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:34.347318 21536 raft_consensus.cc:2229] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:34.347396 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21100
I20260806 06:54:34.347424 21536 raft_consensus.cc:2258] T 5b4bb1e96d044d4e9e323f15f74e0149 P 97162fd780474258aaadc9eb171b107a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:34.349084 21534 ts_tablet_manager.cc:1905] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:34.350665 21534 ts_tablet_manager.cc:1918] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:34.350740 21534 log.cc:1196] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/5b4bb1e96d044d4e9e323f15f74e0149
I20260806 06:54:34.351001 21534 ts_tablet_manager.cc:1939] T 5b4bb1e96d044d4e9e323f15f74e0149 P cc733ded9e3f49b9bf19d2e896749c09: Deleting consensus metadata
I20260806 06:54:34.351413 21023 catalog_manager.cc:4841] TS cc733ded9e3f49b9bf19d2e896749c09 (127.3.247.130:35191): tablet 5b4bb1e96d044d4e9e323f15f74e0149 (table test-table [id=689360bef5394012ac524fd7dd6672c2]) successfully deleted
W20260806 06:54:34.352974 21025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:46623 (error 108)
I20260806 06:54:34.353302 21535 tablet_replica.cc:330] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: stopping tablet replica
I20260806 06:54:34.353418 21535 raft_consensus.cc:2229] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:54:34.353559 21535 raft_consensus.cc:2258] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:54:34.353833 21505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:54:34.353410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:46623 (local address 127.0.0.1:52608)
0806 06:54:34.353457 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.353471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.353520 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.353766 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:46623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 06:54:34.353979 21025 catalog_manager.cc:4584] TS 97162fd780474258aaadc9eb171b107a (127.3.247.129:46623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b4bb1e96d044d4e9e323f15f74e0149: Network error: Client connection negotiation failed: client connection to 127.3.247.129:46623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:54:34.354359 21535 ts_tablet_manager.cc:1905] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:34.356020 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21234
I20260806 06:54:34.356127 21535 ts_tablet_manager.cc:1918] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:34.356196 21535 log.cc:1196] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/5b4bb1e96d044d4e9e323f15f74e0149
I20260806 06:54:34.356454 21535 ts_tablet_manager.cc:1939] T 5b4bb1e96d044d4e9e323f15f74e0149 P e32f85c5aad5426cb82953dd09153186: Deleting consensus metadata
I20260806 06:54:34.356918 21024 catalog_manager.cc:4841] TS e32f85c5aad5426cb82953dd09153186 (127.3.247.131:44619): tablet 5b4bb1e96d044d4e9e323f15f74e0149 (table test-table [id=689360bef5394012ac524fd7dd6672c2]) successfully deleted
I20260806 06:54:34.363512 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21368
I20260806 06:54:34.368404 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21006
2026-08-06T06:54:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2206 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T06:54:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:34Z Disabled control of system clock
I20260806 06:54:34.395454 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36557
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:34.475550 21541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:34.475768 21541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:34.475790 21541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:34.476898 21541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:34.476933 21541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:34.476946 21541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:34.476958 21541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:34.476970 21541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:34.478343 21541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:34.478541 21541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:34.478780 21541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:34.483610 21549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.483654 21547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.483660 21546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:34.483762 21541 server_base.cc:1034] running on GCE node
I20260806 06:54:34.484016 21541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:34.484313 21541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:34.485458 21541 hybrid_clock.cc:648] HybridClock initialized: now 1785999274485446 us; error 28 us; skew 500 ppm
I20260806 06:54:34.486881 21541 webserver.cc:460] Webserver started at http://127.3.247.190:42169/ using document root <none> and password file <none>
I20260806 06:54:34.487133 21541 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:34.487191 21541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:34.487370 21541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:34.488459 21541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "acf4812123aa4ef084f63a813ca0bc8f"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "7d1ed122e9e5579d98bc860e565320bd"
server_key_iv: "0eb5d15c3f7da24e964d9ab2ab855d34"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.488821 21541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "acf4812123aa4ef084f63a813ca0bc8f"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "7d1ed122e9e5579d98bc860e565320bd"
server_key_iv: "0eb5d15c3f7da24e964d9ab2ab855d34"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.490190 21541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:34.490922 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.491079 21541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:34.491180 21541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "acf4812123aa4ef084f63a813ca0bc8f"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "7d1ed122e9e5579d98bc860e565320bd"
server_key_iv: "0eb5d15c3f7da24e964d9ab2ab855d34"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.491261 21541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:34.517469 21541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:34.517881 21541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:34.518054 21541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:34.521802 21541 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:36557
I20260806 06:54:34.521869 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:36557 every 8 connection(s)
I20260806 06:54:34.522248 21541 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:34.523236 21608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:34.525583 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: Bootstrap starting.
I20260806 06:54:34.526222 21608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:34.526537 21608 log.cc:824] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:34.527221 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: No bootstrap required, opened a new log
I20260806 06:54:34.528787 21608 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } }
I20260806 06:54:34.529012 21608 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:34.529058 21608 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf4812123aa4ef084f63a813ca0bc8f, State: Initialized, Role: FOLLOWER
I20260806 06:54:34.529206 21608 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } }
I20260806 06:54:34.529287 21608 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:34.529332 21608 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:34.529379 21608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:34.529748 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21541
I20260806 06:54:34.529882 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:34.530004 21608 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } }
I20260806 06:54:34.530112 4062 external_mini_cluster.cc:1428] Setting key 5734fb08c3cf7db7b296ac247c790a97
I20260806 06:54:34.530120 21608 leader_election.cc:302] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf4812123aa4ef084f63a813ca0bc8f; no voters:
I20260806 06:54:34.530324 21608 leader_election.cc:288] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:34.530407 21611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:34.530593 21611 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [term 1 LEADER]: Becoming Leader. State: Replica: acf4812123aa4ef084f63a813ca0bc8f, State: Running, Role: LEADER
I20260806 06:54:34.530614 21608 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:34.530699 21611 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } }
I20260806 06:54:34.530997 21611 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf4812123aa4ef084f63a813ca0bc8f. Latest consensus state: current_term: 1 leader_uuid: "acf4812123aa4ef084f63a813ca0bc8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } } }
I20260806 06:54:34.531080 21611 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:34.531160 21612 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf4812123aa4ef084f63a813ca0bc8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf4812123aa4ef084f63a813ca0bc8f" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36557 } } }
I20260806 06:54:34.531244 21612 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:34.531800 21620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:34.532423 21620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:34.534008 21620 catalog_manager.cc:1347] Generated new cluster ID: 4a072f4c8ea0476380005247a54b8e38
I20260806 06:54:34.534060 21620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:34.536689 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.530847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.3.247.190:36557)
0806 06:54:34.531071 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.531082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.531403 (+ 321us) server_negotiation.cc:407] Connection header received
0806 06:54:34.531557 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.531565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.531654 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.532044 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:34.532191 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.533420 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.534573 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.535157 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.535268 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.535983 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:34.535987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:34.535987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:34.536185 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:34.536246 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:34.536261 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:34.536320 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.536419 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.536492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":60,"threads_started":1}
I20260806 06:54:34.541144 21620 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:34.541639 21620 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:34.550153 21620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: Generated new TSK 0
I20260806 06:54:34.550375 21620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:34.552875 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--builtin_ntp_servers=127.3.247.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:34.635582 21632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:34.635804 21632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:34.635828 21632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:34.637007 21632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:34.637048 21632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:34.637074 21632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:34.638513 21632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:34.638758 21632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:34.639081 21632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:34.644820 21637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.644968 21640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.644966 21638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:34.645365 21632 server_base.cc:1034] running on GCE node
I20260806 06:54:34.645612 21632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:34.645915 21632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:34.647089 21632 hybrid_clock.cc:648] HybridClock initialized: now 1785999274647070 us; error 35 us; skew 500 ppm
I20260806 06:54:34.648499 21632 webserver.cc:460] Webserver started at http://127.3.247.129:42217/ using document root <none> and password file <none>
I20260806 06:54:34.648803 21632 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:34.648873 21632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:34.649017 21632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:34.650218 21632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "9d7a3d49f2f74befa6d9914be847a4ee"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "95a1d486a5161f83033d9e17773106b9"
server_key_iv: "baa39c946035f39f4b23f28b67d5e930"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.650581 21632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "9d7a3d49f2f74befa6d9914be847a4ee"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "95a1d486a5161f83033d9e17773106b9"
server_key_iv: "baa39c946035f39f4b23f28b67d5e930"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.651981 21632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:34.652787 21646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.652943 21632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:34.653033 21632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "9d7a3d49f2f74befa6d9914be847a4ee"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "95a1d486a5161f83033d9e17773106b9"
server_key_iv: "baa39c946035f39f4b23f28b67d5e930"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.653105 21632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:34.673971 21632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:34.674324 21632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:34.674477 21632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:34.674722 21632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:34.675087 21632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:34.675148 21632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.675186 21632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:34.675277 21632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.681509 21632 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:37867
I20260806 06:54:34.681555 21759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:37867 every 8 connection(s)
I20260806 06:54:34.681967 21632 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:54:34.687024 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.682673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49977 (local address 127.3.247.190:36557)
0806 06:54:34.682748 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.682749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.683014 (+ 265us) server_negotiation.cc:407] Connection header received
0806 06:54:34.683494 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.683495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.683505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.683556 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:34.684235 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.684737 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.685687 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.685951 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.686007 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.686803 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:34.686805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:34.686805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:34.686809 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:34.686821 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:34.686825 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:34.686863 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.686934 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.686959 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:54:34.687248 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.682690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36557 (local address 127.3.247.129:49977)
0806 06:54:34.682884 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.682919 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.683400 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.683609 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.683612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.683620 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:34.684161 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.684165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.684825 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.684826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.685611 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.685613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.686026 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.686027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.686142 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.686644 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:34.686644 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:34.686674 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:34.686699 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:34.686870 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:34.686873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:34.686875 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.686922 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.687001 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260806 06:54:34.687839 21760 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36557
I20260806 06:54:34.687968 21760 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:34.688122 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21632
I20260806 06:54:34.688212 21760 heartbeater.cc:511] Master 127.3.247.190:36557 requested a full tablet report, sending...
I20260806 06:54:34.688263 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:34.688458 4062 external_mini_cluster.cc:1428] Setting key bf8bfeac8f3c35a92917b43d5d1b2c93
I20260806 06:54:34.688802 21572 ts_manager.cc:194] Registered new tserver with Master: 9d7a3d49f2f74befa6d9914be847a4ee (127.3.247.129:37867)
I20260806 06:54:34.689533 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--builtin_ntp_servers=127.3.247.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:54:34.689569 21572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.129:49977
W20260806 06:54:34.772140 21763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:34.772413 21763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:34.772450 21763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:34.773610 21763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:34.773686 21763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:34.773730 21763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:34.775123 21763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:34.775409 21763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:34.775722 21763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:34.781200 21769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.781208 21768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.781232 21771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:34.781392 21763 server_base.cc:1034] running on GCE node
I20260806 06:54:34.781658 21763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:34.782017 21763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:34.783294 21763 hybrid_clock.cc:648] HybridClock initialized: now 1785999274783270 us; error 46 us; skew 500 ppm
I20260806 06:54:34.784845 21763 webserver.cc:460] Webserver started at http://127.3.247.130:42409/ using document root <none> and password file <none>
I20260806 06:54:34.785110 21763 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:34.785163 21763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:34.785324 21763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:34.786511 21763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "1e419996f85f42e895c8570bea2d00d6"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "f3d5aeb7dbdfcefd13701684093caca1"
server_key_iv: "ec88baf5eecdba1f77df4f7f44b302fe"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.786871 21763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "1e419996f85f42e895c8570bea2d00d6"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "f3d5aeb7dbdfcefd13701684093caca1"
server_key_iv: "ec88baf5eecdba1f77df4f7f44b302fe"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.788555 21763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:34.789345 21777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.789511 21763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:34.789620 21763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "1e419996f85f42e895c8570bea2d00d6"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "f3d5aeb7dbdfcefd13701684093caca1"
server_key_iv: "ec88baf5eecdba1f77df4f7f44b302fe"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.789698 21763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:34.803169 21763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:34.803519 21763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:34.803689 21763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:34.803937 21763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:34.804306 21763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:34.804363 21763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.804407 21763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:34.804440 21763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.810115 21763 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:37769
I20260806 06:54:34.810216 21890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:37769 every 8 connection(s)
I20260806 06:54:34.810567 21763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:54:34.814298 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21763
I20260806 06:54:34.814420 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:34.814601 4062 external_mini_cluster.cc:1428] Setting key d9ff849df1f5e4d7395a3cae2316868b
I20260806 06:54:34.815668 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--builtin_ntp_servers=127.3.247.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:54:34.816727 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.811373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:60085 (local address 127.3.247.190:36557)
0806 06:54:34.811464 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.811467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.811669 (+ 202us) server_negotiation.cc:407] Connection header received
0806 06:54:34.812375 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.812378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.812396 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.812462 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:34.813446 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.814019 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.815283 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.815513 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.815568 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.816501 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:34.816502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:34.816503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:34.816507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:34.816520 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:34.816524 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:34.816562 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.816643 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.816667 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:54:34.817049 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.811338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36557 (local address 127.3.247.130:60085)
0806 06:54:34.811540 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.811576 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.812207 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.812548 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.812553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.812563 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:34.813349 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.813355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.814108 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.814109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.815178 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.815182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.815608 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.815609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.815792 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.816354 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:34.816355 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:34.816396 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:34.816427 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:34.816572 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:34.816576 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:34.816579 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.816647 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.816769 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260806 06:54:34.817735 21891 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36557
I20260806 06:54:34.817873 21891 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:34.818065 21891 heartbeater.cc:511] Master 127.3.247.190:36557 requested a full tablet report, sending...
I20260806 06:54:34.818427 21572 ts_manager.cc:194] Registered new tserver with Master: 1e419996f85f42e895c8570bea2d00d6 (127.3.247.130:37769)
I20260806 06:54:34.818804 21572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.130:60085
W20260806 06:54:34.898703 21894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:34.898921 21894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:34.898943 21894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:34.900134 21894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:34.900180 21894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:34.900206 21894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:34.901592 21894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:36557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:34.901901 21894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:34.902238 21894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:34.907522 21902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.907522 21900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:34.907523 21899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:34.907542 21894 server_base.cc:1034] running on GCE node
I20260806 06:54:34.907981 21894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:34.908263 21894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:34.909446 21894 hybrid_clock.cc:648] HybridClock initialized: now 1785999274909441 us; error 37 us; skew 500 ppm
I20260806 06:54:34.910943 21894 webserver.cc:460] Webserver started at http://127.3.247.131:36811/ using document root <none> and password file <none>
I20260806 06:54:34.911237 21894 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:34.911310 21894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:34.911463 21894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:34.912567 21894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "12ba5ce6bac24c5aaa89f3cc95efffca"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "e561d6ed24109e5512b7ffbd6bf1bfe3"
server_key_iv: "be4ae87e41139124b4dd4dc6cb001078"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.912935 21894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "12ba5ce6bac24c5aaa89f3cc95efffca"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "e561d6ed24109e5512b7ffbd6bf1bfe3"
server_key_iv: "be4ae87e41139124b4dd4dc6cb001078"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.914368 21894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:34.915077 21908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.915303 21894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:34.915400 21894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "12ba5ce6bac24c5aaa89f3cc95efffca"
format_stamp: "Formatted at 2026-08-06 06:54:34 on dist-test-slave-0kls"
server_key: "e561d6ed24109e5512b7ffbd6bf1bfe3"
server_key_iv: "be4ae87e41139124b4dd4dc6cb001078"
server_key_version: "encryptionkey@0"
I20260806 06:54:34.915480 21894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:34.938767 21894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:34.939129 21894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:34.939296 21894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:34.939545 21894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:34.939877 21894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:34.939929 21894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.939970 21894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:34.939997 21894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:34.945750 21894 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:46539
I20260806 06:54:34.945843 22021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:46539 every 8 connection(s)
I20260806 06:54:34.946159 21894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:34.951511 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 21894
I20260806 06:54:34.951548 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.946918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:46409 (local address 127.3.247.190:36557)
0806 06:54:34.946995 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.946997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.947261 (+ 264us) server_negotiation.cc:407] Connection header received
0806 06:54:34.947796 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.947797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.947810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.947856 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:34.948547 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.949057 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.950072 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.950286 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.950364 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.951258 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:34.951259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:34.951260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:34.951265 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:34.951279 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:34.951284 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:34.951334 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.951435 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.951465 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:54:34.951632 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:34.951746 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.946901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36557 (local address 127.3.247.131:46409)
0806 06:54:34.947108 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.947144 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.947690 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.947925 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.947929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.947937 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:34.948446 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.948450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.949161 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.949162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.949978 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.949980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.950382 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.950383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.950507 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.951085 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:34.951085 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:34.951115 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:34.951138 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:34.951348 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:34.951353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:34.951356 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.951436 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.951538 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260806 06:54:34.951853 4062 external_mini_cluster.cc:1428] Setting key cf4bfcc70e3ab47f389dd59741db95c9
I20260806 06:54:34.952385 22022 heartbeater.cc:347] Connected to a master server at 127.3.247.190:36557
I20260806 06:54:34.952505 22022 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:34.952697 22022 heartbeater.cc:511] Master 127.3.247.190:36557 requested a full tablet report, sending...
I20260806 06:54:34.953074 21572 ts_manager.cc:194] Registered new tserver with Master: 12ba5ce6bac24c5aaa89f3cc95efffca (127.3.247.131:46539)
I20260806 06:54:34.953408 21572 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.131:46409
I20260806 06:54:34.964007 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:34.968144 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.965414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.3.247.190:36557)
0806 06:54:34.965489 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.965491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.965535 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:54:34.965599 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.965599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.965608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.965655 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:34.965777 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.966348 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.967047 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.967217 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.967266 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.967915 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:34.967916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:34.967917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:34.967920 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:34.967932 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:34.967935 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:34.967972 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.968009 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.968025 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:54:34.969877 21572 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:34.971489 21572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:34.977883 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.973736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.3.247.130:37769)
0806 06:54:34.974024 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.974028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.974180 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:54:34.974240 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.974243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.974259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.974338 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:34.974569 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975316 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.976691 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977435 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977512 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977609 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.977722 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.977763 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260806 06:54:34.977907 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.973637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:37769 (local address 127.0.0.1:45744)
0806 06:54:34.974036 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.974047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.974104 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.974341 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.974344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.974346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:34.974486 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.974488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975404 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.975405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.976595 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.976597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977526 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.977605 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977617 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.977691 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.977764 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":7,"thread_start_us":103,"threads_started":1}
I20260806 06:54:34.978130 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.973629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37867 (local address 127.0.0.1:50120)
0806 06:54:34.973968 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.974012 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.974092 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.974382 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.974383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.974385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:34.974474 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.974477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975778 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.975779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.976888 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.976890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977873 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.977943 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977947 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.978003 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.978044 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":22,"thread_start_us":85,"threads_started":1}
I20260806 06:54:34.978348 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.973909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:46539 (local address 127.0.0.1:38592)
0806 06:54:34.974185 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:34.974196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:34.974246 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:34.974460 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:34.974460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:34.974462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:34.974500 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.974501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975793 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.975794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.976690 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:34.976692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977669 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:34.977723 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977726 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:34.977770 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:54:34.977872 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260806 06:54:34.978407 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.973750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50120 (local address 127.3.247.129:37867)
0806 06:54:34.973933 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.973938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.974074 (+ 136us) server_negotiation.cc:407] Connection header received
0806 06:54:34.974208 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.974210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.974226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.974310 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:34.974559 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975687 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.976975 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977662 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977739 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977837 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.978291 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.978332 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260806 06:54:34.978597 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:34.974021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.3.247.131:46539)
0806 06:54:34.974178 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:34.974182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:34.974241 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:54:34.974317 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:34.974320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:34.974333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:34.974398 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:34.974574 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.975687 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.976766 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:34.977457 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:34.977544 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:34.977636 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:34.978453 (+ 817us) server_negotiation.cc:299] Negotiation successful
0806 06:54:34.978501 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260806 06:54:34.978688 21825 tablet_service.cc:1467] Processing CreateTablet for tablet 865e9eee2de14c2497b04c59778312cd (DEFAULT_TABLE table=test-table [id=0fc79edc828346b1a0f163b0790571a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:34.979089 21825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865e9eee2de14c2497b04c59778312cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:34.979260 21956 tablet_service.cc:1467] Processing CreateTablet for tablet 865e9eee2de14c2497b04c59778312cd (DEFAULT_TABLE table=test-table [id=0fc79edc828346b1a0f163b0790571a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:34.979552 21956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865e9eee2de14c2497b04c59778312cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:34.981817 22036 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Bootstrap starting.
I20260806 06:54:34.982128 22037 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Bootstrap starting.
I20260806 06:54:34.982398 21694 tablet_service.cc:1467] Processing CreateTablet for tablet 865e9eee2de14c2497b04c59778312cd (DEFAULT_TABLE table=test-table [id=0fc79edc828346b1a0f163b0790571a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:34.982703 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865e9eee2de14c2497b04c59778312cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:34.982817 22037 tablet_bootstrap.cc:654] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:34.983093 22037 log.cc:824] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:34.983779 22037 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: No bootstrap required, opened a new log
I20260806 06:54:34.983862 22037 ts_tablet_manager.cc:1397] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:34.985220 22039 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Bootstrap starting.
I20260806 06:54:34.985893 22039 tablet_bootstrap.cc:654] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:34.985893 22036 tablet_bootstrap.cc:654] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:34.986231 22036 log.cc:824] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:34.986246 22039 log.cc:824] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:34.986184 22037 raft_consensus.cc:348] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.986342 22037 raft_consensus.cc:374] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:34.986382 22037 raft_consensus.cc:729] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ba5ce6bac24c5aaa89f3cc95efffca, State: Initialized, Role: FOLLOWER
I20260806 06:54:34.986493 22037 consensus_queue.cc:260] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.986749 22037 ts_tablet_manager.cc:1428] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:54:34.986847 22022 heartbeater.cc:503] Master 127.3.247.190:36557 was elected leader, sending a full tablet report...
I20260806 06:54:34.987478 22036 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: No bootstrap required, opened a new log
I20260806 06:54:34.987555 22036 ts_tablet_manager.cc:1397] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260806 06:54:34.987617 22039 tablet_bootstrap.cc:492] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: No bootstrap required, opened a new log
I20260806 06:54:34.987771 22039 ts_tablet_manager.cc:1397] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:54:34.990283 22039 raft_consensus.cc:348] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.990283 22036 raft_consensus.cc:348] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.990458 22036 raft_consensus.cc:374] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:34.990458 22039 raft_consensus.cc:374] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:34.990495 22036 raft_consensus.cc:729] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e419996f85f42e895c8570bea2d00d6, State: Initialized, Role: FOLLOWER
I20260806 06:54:34.990495 22039 raft_consensus.cc:729] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7a3d49f2f74befa6d9914be847a4ee, State: Initialized, Role: FOLLOWER
I20260806 06:54:34.990617 22036 consensus_queue.cc:260] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.990617 22039 consensus_queue.cc:260] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:34.990895 22039 ts_tablet_manager.cc:1428] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:54:34.990944 21760 heartbeater.cc:503] Master 127.3.247.190:36557 was elected leader, sending a full tablet report...
I20260806 06:54:34.990895 22036 ts_tablet_manager.cc:1428] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 06:54:34.991119 21891 heartbeater.cc:503] Master 127.3.247.190:36557 was elected leader, sending a full tablet report...
W20260806 06:54:35.061203 21892 tablet.cc:2369] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:54:35.182727 21761 tablet.cc:2369] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:54:35.196741 22023 tablet.cc:2369] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:35.213551 22044 raft_consensus.cc:482] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:35.213691 22044 raft_consensus.cc:504] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:35.214161 22044 leader_election.cc:288] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e419996f85f42e895c8570bea2d00d6 (127.3.247.130:37769), 12ba5ce6bac24c5aaa89f3cc95efffca (127.3.247.131:46539)
I20260806 06:54:35.217281 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.214457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49125 (local address 127.3.247.130:37769)
0806 06:54:35.214506 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:35.214508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:35.214646 (+ 138us) server_negotiation.cc:407] Connection header received
0806 06:54:35.214781 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:35.214782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:35.214795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:35.214853 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:35.215150 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.215583 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.216575 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.216935 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.216991 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:35.216997 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:35.217168 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:54:35.217215 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:54:35.217290 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.214380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:37769 (local address 127.3.247.129:49125)
0806 06:54:35.214580 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:35.214598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:35.214686 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:35.214878 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:35.214879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:35.214881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:35.215072 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:35.215074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.215686 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.215687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:35.216480 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:35.216482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.217016 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.217017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:35.217095 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:35.217098 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:35.217146 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:54:35.217178 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":12,"thread_start_us":161,"threads_started":1}
I20260806 06:54:35.217343 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.214509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:55979 (local address 127.3.247.131:46539)
0806 06:54:35.214556 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:35.214558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:35.214610 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:54:35.214711 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:35.214712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:35.214724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:35.214779 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:35.215057 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.215564 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.216596 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.216954 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.217044 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:35.217052 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:35.217230 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:54:35.217269 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:54:35.217576 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.214378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:46539 (local address 127.3.247.129:55979)
0806 06:54:35.214548 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:35.214560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:35.214627 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:35.214802 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:35.214803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:35.214806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:35.214883 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:35.214885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.215648 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.215650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:35.216484 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:35.216486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.217072 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.217073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:35.217148 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:35.217153 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:54:35.217210 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:54:35.217245 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260806 06:54:35.217768 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865e9eee2de14c2497b04c59778312cd" candidate_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e419996f85f42e895c8570bea2d00d6" is_pre_election: true
I20260806 06:54:35.217974 21845 raft_consensus.cc:2454] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7a3d49f2f74befa6d9914be847a4ee in term 0.
I20260806 06:54:35.217990 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865e9eee2de14c2497b04c59778312cd" candidate_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" is_pre_election: true
I20260806 06:54:35.218120 21976 raft_consensus.cc:2454] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7a3d49f2f74befa6d9914be847a4ee in term 0.
I20260806 06:54:35.218209 21647 leader_election.cc:302] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e419996f85f42e895c8570bea2d00d6, 9d7a3d49f2f74befa6d9914be847a4ee; no voters:
I20260806 06:54:35.218361 22044 raft_consensus.cc:2781] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:35.218425 22044 raft_consensus.cc:482] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:35.218443 22044 raft_consensus.cc:3037] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:35.219120 22044 raft_consensus.cc:504] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:35.219322 22044 leader_election.cc:288] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [CANDIDATE]: Term 1 election: Requested vote from peers 1e419996f85f42e895c8570bea2d00d6 (127.3.247.130:37769), 12ba5ce6bac24c5aaa89f3cc95efffca (127.3.247.131:46539)
I20260806 06:54:35.219403 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865e9eee2de14c2497b04c59778312cd" candidate_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca"
I20260806 06:54:35.219419 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865e9eee2de14c2497b04c59778312cd" candidate_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e419996f85f42e895c8570bea2d00d6"
I20260806 06:54:35.219491 21845 raft_consensus.cc:3037] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:35.219491 21976 raft_consensus.cc:3037] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:35.220471 21976 raft_consensus.cc:2454] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7a3d49f2f74befa6d9914be847a4ee in term 1.
I20260806 06:54:35.220471 21845 raft_consensus.cc:2454] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7a3d49f2f74befa6d9914be847a4ee in term 1.
I20260806 06:54:35.220669 21647 leader_election.cc:302] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e419996f85f42e895c8570bea2d00d6, 9d7a3d49f2f74befa6d9914be847a4ee; no voters:
I20260806 06:54:35.220778 22044 raft_consensus.cc:2781] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:35.220949 22044 raft_consensus.cc:686] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [term 1 LEADER]: Becoming Leader. State: Replica: 9d7a3d49f2f74befa6d9914be847a4ee, State: Running, Role: LEADER
I20260806 06:54:35.221062 22044 consensus_queue.cc:237] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } }
I20260806 06:54:35.221961 21571 catalog_manager.cc:5495] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7a3d49f2f74befa6d9914be847a4ee (127.3.247.129). New cstate: current_term: 1 leader_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7a3d49f2f74befa6d9914be847a4ee" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 } health_report { overall_health: UNKNOWN } } }
I20260806 06:54:35.264159 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.261943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.3.247.129:37867)
0806 06:54:35.261997 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:35.261999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:35.262008 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:54:35.262048 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:35.262049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:35.262070 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:35.262139 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:35.262239 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.262748 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.263466 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:35.263688 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:35.263745 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:35.263865 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:35.263969 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:35.264029 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:35.264076 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:54:35.264093 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:54:35.266402 21845 raft_consensus.cc:1261] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 1 FOLLOWER]: Refusing update from remote peer 9d7a3d49f2f74befa6d9914be847a4ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:35.266402 21976 raft_consensus.cc:1261] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 1 FOLLOWER]: Refusing update from remote peer 9d7a3d49f2f74befa6d9914be847a4ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:35.266803 22044 consensus_queue.cc:1035] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ba5ce6bac24c5aaa89f3cc95efffca" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 46539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:35.267019 22047 consensus_queue.cc:1035] T 865e9eee2de14c2497b04c59778312cd P 9d7a3d49f2f74befa6d9914be847a4ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e419996f85f42e895c8570bea2d00d6" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:35.269423 22050 mvcc.cc:205] Tried to move back new op lower bound from 7315453031487987712 to 7315453031306276864. Current Snapshot: MvccSnapshot[applied={T|T < 7315453031487987712}]
I20260806 06:54:35.270025 22051 mvcc.cc:205] Tried to move back new op lower bound from 7315453031487987712 to 7315453031306276864. Current Snapshot: MvccSnapshot[applied={T|T < 7315453031487987712}]
I20260806 06:54:35.270620 22052 mvcc.cc:205] Tried to move back new op lower bound from 7315453031487987712 to 7315453031306276864. Current Snapshot: MvccSnapshot[applied={T|T < 7315453031487987712 or (T in {7315453031487987712})}]
I20260806 06:54:35.271708 21572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:35.271816 21572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:35.272607 21572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acf4812123aa4ef084f63a813ca0bc8f: Sending DeleteTablet for 3 replicas of tablet 865e9eee2de14c2497b04c59778312cd
I20260806 06:54:35.272990 21825 tablet_service.cc:1514] Processing DeleteTablet for tablet 865e9eee2de14c2497b04c59778312cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:35 UTC) from {username='slave'} at 127.0.0.1:45744
I20260806 06:54:35.273239 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21632
I20260806 06:54:35.273216 21956 tablet_service.cc:1514] Processing DeleteTablet for tablet 865e9eee2de14c2497b04c59778312cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:35 UTC) from {username='slave'} at 127.0.0.1:38592
I20260806 06:54:35.274202 22060 tablet_replica.cc:330] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: stopping tablet replica
I20260806 06:54:35.274372 22060 raft_consensus.cc:2229] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:35.274488 22060 raft_consensus.cc:2258] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:35.274886 22060 ts_tablet_manager.cc:1905] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:35.276840 22060 ts_tablet_manager.cc:1918] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:35.276918 22060 log.cc:1196] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/865e9eee2de14c2497b04c59778312cd
I20260806 06:54:35.277158 22060 ts_tablet_manager.cc:1939] T 865e9eee2de14c2497b04c59778312cd P 1e419996f85f42e895c8570bea2d00d6: Deleting consensus metadata
I20260806 06:54:35.277585 21556 catalog_manager.cc:4841] TS 1e419996f85f42e895c8570bea2d00d6 (127.3.247.130:37769): tablet 865e9eee2de14c2497b04c59778312cd (table test-table [id=0fc79edc828346b1a0f163b0790571a1]) successfully deleted
W20260806 06:54:35.279198 21557 connection.cc:535] client connection to 127.3.247.129:37867 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:54:35.279299 21557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:54:35.279461 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21763
I20260806 06:54:35.280280 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.279659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37867 (local address 127.0.0.1:50130)
0806 06:54:35.280194 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:35.280240 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510}
W20260806 06:54:35.280424 21557 catalog_manager.cc:4584] TS 9d7a3d49f2f74befa6d9914be847a4ee (127.3.247.129:37867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 865e9eee2de14c2497b04c59778312cd: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37867: connect: Connection refused (error 111)
I20260806 06:54:35.282315 22061 tablet_replica.cc:330] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: stopping tablet replica
I20260806 06:54:35.282466 22061 raft_consensus.cc:2229] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:35.282610 22061 raft_consensus.cc:2258] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:35.283087 22061 ts_tablet_manager.cc:1905] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:35.284626 22061 ts_tablet_manager.cc:1918] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:35.284713 22061 log.cc:1196] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/865e9eee2de14c2497b04c59778312cd
I20260806 06:54:35.284951 22061 ts_tablet_manager.cc:1939] T 865e9eee2de14c2497b04c59778312cd P 12ba5ce6bac24c5aaa89f3cc95efffca: Deleting consensus metadata
I20260806 06:54:35.284998 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21894
I20260806 06:54:35.289743 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:35.289591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:46539 (local address 127.0.0.1:38602)
0806 06:54:35.289656 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:35.289692 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.131:46539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:54:35.290037 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 21541
2026-08-06T06:54:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (921 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:35 dist-test-slave-0kls krb5kdc[22065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:35 dist-test-slave-0kls krb5kdc[22065](info): set up 2 sockets
Aug 06 06:54:35 dist-test-slave-0kls krb5kdc[22065](info): commencing operation
krb5kdc: starting...
W20260806 06:54:36.313853 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:54:36.436895 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:45415
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:45415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:36.519160 22078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:36.519381 22078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:36.519404 22078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:36.520504 22078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:36.520539 22078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:36.520553 22078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:36.520566 22078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:36.520577 22078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:36.521999 22078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:45415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:45415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:36.522194 22078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:36.522373 22078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:36.527206 22083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:36.527276 22086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:36.527210 22084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:36.527494 22078 server_base.cc:1034] running on GCE node
I20260806 06:54:36.527793 22078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:36.528075 22078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:36.529232 22078 hybrid_clock.cc:648] HybridClock initialized: now 1785999276529220 us; error 32 us; skew 500 ppm
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:36.531810 22078 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:36.532246 22078 webserver.cc:460] Webserver started at http://127.3.247.190:39535/ using document root <none> and password file <none>
I20260806 06:54:36.532512 22078 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:36.532559 22078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:36.532704 22078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:36.533874 22078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "edc3edc0e61845df9cf4fa4b4d315a91"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "46160adecdd446418a746e008f4243e9"
server_key_iv: "0ab49384d9676ae5d6b801fc8e3142c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.534258 22078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "edc3edc0e61845df9cf4fa4b4d315a91"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "46160adecdd446418a746e008f4243e9"
server_key_iv: "0ab49384d9676ae5d6b801fc8e3142c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.535626 22078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:54:36.536389 22093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.536573 22078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:36.536664 22078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "edc3edc0e61845df9cf4fa4b4d315a91"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "46160adecdd446418a746e008f4243e9"
server_key_iv: "0ab49384d9676ae5d6b801fc8e3142c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.536721 22078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:36.555980 22078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:36.558387 22078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:36.558516 22078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:36.562757 22078 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:45415
I20260806 06:54:36.562819 22145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:45415 every 8 connection(s)
I20260806 06:54:36.563192 22078 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:36.564227 22146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:36.566499 22146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: Bootstrap starting.
I20260806 06:54:36.567040 22146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:36.567270 22146 log.cc:824] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:36.567955 22146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: No bootstrap required, opened a new log
I20260806 06:54:36.569490 22146 raft_consensus.cc:348] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } }
I20260806 06:54:36.569629 22146 raft_consensus.cc:374] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:36.569651 22146 raft_consensus.cc:729] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc3edc0e61845df9cf4fa4b4d315a91, State: Initialized, Role: FOLLOWER
I20260806 06:54:36.569825 22146 consensus_queue.cc:260] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } }
I20260806 06:54:36.569912 22146 raft_consensus.cc:388] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:36.569936 22146 raft_consensus.cc:482] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:36.569965 22146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:36.570444 22146 raft_consensus.cc:504] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } }
I20260806 06:54:36.570526 22146 leader_election.cc:302] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edc3edc0e61845df9cf4fa4b4d315a91; no voters:
I20260806 06:54:36.570650 22146 leader_election.cc:288] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:36.570694 22149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:36.570953 22146 sys_catalog.cc:564] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:36.570971 22149 raft_consensus.cc:686] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [term 1 LEADER]: Becoming Leader. State: Replica: edc3edc0e61845df9cf4fa4b4d315a91, State: Running, Role: LEADER
I20260806 06:54:36.571272 22149 consensus_queue.cc:237] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } }
I20260806 06:54:36.571731 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edc3edc0e61845df9cf4fa4b4d315a91. Latest consensus state: current_term: 1 leader_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } } }
I20260806 06:54:36.571841 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:36.572176 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22078
I20260806 06:54:36.572302 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:36.571857 22150 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc3edc0e61845df9cf4fa4b4d315a91" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 45415 } } }
I20260806 06:54:36.572381 22150 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:36.572542 4062 external_mini_cluster.cc:1428] Setting key 6c3c20f4e7fe6c6ba05e442aa56869c3
W20260806 06:54:36.573068 22165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:54:36.573189 22165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:54:36.573277 22158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:36.573452 22158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:36.575166 22158 catalog_manager.cc:1347] Generated new cluster ID: 056f1e0fb8504ba0a8eb6f4f03b64d65
I20260806 06:54:36.575220 22158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:36.583499 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:36.573063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.3.247.190:45415)
0806 06:54:36.573417 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:36.573427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:36.573468 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:54:36.573605 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:36.573616 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:36.573664 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:36.573833 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:36.574817 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.575881 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.576706 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.577148 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.577285 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:36.579819 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:36.579823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:36.579823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:36.579837 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:36.581715 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.582160 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.582163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.582163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.582285 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.582538 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.582539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.582539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.582808 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:36.582993 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:36.583073 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:54:36.583242 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":80,"threads_started":1}
I20260806 06:54:36.584481 22158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:36.585021 22158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:36.599761 22158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: Generated new TSK 0
I20260806 06:54:36.600014 22158 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:54:36.630780 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:36.714267 22172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:36.714490 22172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:36.714531 22172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:36.715732 22172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:36.715790 22172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:36.715834 22172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:36.717275 22172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:36.717535 22172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:36.717773 22172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:36.722733 22180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:36.722761 22178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:36.722761 22177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:36.722891 22172 server_base.cc:1034] running on GCE node
I20260806 06:54:36.723080 22172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:36.723294 22172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:36.724454 22172 hybrid_clock.cc:648] HybridClock initialized: now 1785999276724448 us; error 36 us; skew 500 ppm
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:36.727089 22172 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:54:36.727527 22172 webserver.cc:460] Webserver started at http://127.3.247.129:36015/ using document root <none> and password file <none>
I20260806 06:54:36.727782 22172 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:36.727855 22172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:36.728008 22172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:36.729077 22172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "70425d0d222e4e30b4b88b02bb8042e2"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "796250eeb16bc682ec31732b4c420b94"
server_key_iv: "b559aa31c07680ee4213bbb5ec8015c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.729439 22172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "70425d0d222e4e30b4b88b02bb8042e2"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "796250eeb16bc682ec31732b4c420b94"
server_key_iv: "b559aa31c07680ee4213bbb5ec8015c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.730834 22172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:36.731664 22187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.731926 22172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:54:36.732033 22172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "70425d0d222e4e30b4b88b02bb8042e2"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "796250eeb16bc682ec31732b4c420b94"
server_key_iv: "b559aa31c07680ee4213bbb5ec8015c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.732120 22172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:36.745244 22172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:36.747421 22172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:36.747591 22172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:36.747833 22172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:36.748144 22172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:36.748215 22172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.748268 22172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:36.748317 22172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.756101 22172 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:37507
I20260806 06:54:36.756146 22300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:37507 every 8 connection(s)
I20260806 06:54:36.756507 22172 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:36.765848 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:36.757231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:37027 (local address 127.3.247.190:45415)
0806 06:54:36.757277 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:36.757279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:36.757491 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:54:36.758090 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:36.758091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:36.758105 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:36.758159 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:36.759085 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.759591 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.760573 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.760767 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.760824 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:36.763118 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:36.763120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:36.763120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:36.763127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:36.764565 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.764969 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.764971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.764971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.765022 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.765241 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.765242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.765242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.765372 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:36.765422 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:36.765660 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 06:54:36.765724 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:54:36.766080 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:36.757221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:45415 (local address 127.3.247.129:37027)
0806 06:54:36.757381 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:36.757420 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:36.757988 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:36.758215 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:36.758218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:36.758797 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:36.759000 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:36.759005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.759676 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.759677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:36.760482 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:36.760485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.760847 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.760848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:36.760950 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:36.761437 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:36.761445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:36.763022 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:36.764644 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:36.764657 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:36.764659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:36.764899 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:36.765083 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:36.765084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:36.765084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:36.765186 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:36.765427 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:36.765431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:36.765550 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 06:54:36.765632 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:54:36.765753 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260806 06:54:36.766443 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22172
I20260806 06:54:36.766566 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:36.766664 22301 heartbeater.cc:347] Connected to a master server at 127.3.247.190:45415
I20260806 06:54:36.766773 22301 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:36.766824 4062 external_mini_cluster.cc:1428] Setting key 53487ac49b41eca8c61b5901666821be
I20260806 06:54:36.766986 22301 heartbeater.cc:511] Master 127.3.247.190:45415 requested a full tablet report, sending...
I20260806 06:54:36.767675 22110 ts_manager.cc:194] Registered new tserver with Master: 70425d0d222e4e30b4b88b02bb8042e2 (127.3.247.129:37507)
I20260806 06:54:36.768290 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:37027
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:54:36.792083 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:36.872668 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:36.872913 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:36.872949 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:36.874176 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:36.874246 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:36.874289 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:36.875737 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:36.876016 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:36.876271 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:36.881515 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:36.881507 22314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:36.881551 22306 server_base.cc:1034] running on GCE node
W20260806 06:54:36.881507 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:36.882118 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:36.882378 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:36.883548 22306 hybrid_clock.cc:648] HybridClock initialized: now 1785999276883544 us; error 37 us; skew 500 ppm
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:36.886053 22306 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:54:36.886576 22306 webserver.cc:460] Webserver started at http://127.3.247.130:45363/ using document root <none> and password file <none>
I20260806 06:54:36.886827 22306 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:36.886898 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:36.887046 22306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:36.888113 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "742f4d89e8b240d9ae5b7433649b4af9"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "649a43e1ea5b71fcf65e3f916fa6e792"
server_key_iv: "68bb931e24c5c8aded7ceaf5b154decd"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.888481 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "742f4d89e8b240d9ae5b7433649b4af9"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "649a43e1ea5b71fcf65e3f916fa6e792"
server_key_iv: "68bb931e24c5c8aded7ceaf5b154decd"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.889827 22306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:36.890599 22321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.890761 22306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:36.890853 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "742f4d89e8b240d9ae5b7433649b4af9"
format_stamp: "Formatted at 2026-08-06 06:54:36 on dist-test-slave-0kls"
server_key: "649a43e1ea5b71fcf65e3f916fa6e792"
server_key_iv: "68bb931e24c5c8aded7ceaf5b154decd"
server_key_version: "encryptionkey@0"
I20260806 06:54:36.890935 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:36.921175 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:36.923123 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:36.923302 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:36.923568 22306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:36.923890 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:36.923946 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.923992 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:36.924015 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:36.929816 22306 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:34909
I20260806 06:54:36.929857 22434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:34909 every 8 connection(s)
I20260806 06:54:36.930184 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:54:36 dist-test-slave-0kls krb5kdc[22065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999276, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:36.937036 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22306
I20260806 06:54:36.937193 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:36.937425 4062 external_mini_cluster.cc:1428] Setting key 4eb069cbc0715bd6dc7415bb458ccdb8
I20260806 06:54:36.940541 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:36.930979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:39225 (local address 127.3.247.190:45415)
0806 06:54:36.931071 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:36.931074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:36.931269 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:54:36.931883 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:36.931885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:36.931899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:36.931954 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:36.932964 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.933500 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.934548 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.934749 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.934843 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:36.937101 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:36.937102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:36.937103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:36.937117 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:36.939002 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.939448 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.939449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.939450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.939518 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:36.939814 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:36.939815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:36.939816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:36.939974 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:36.940055 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:36.940406 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 06:54:36.940477 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:54:36.940855 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:36.930970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:45415 (local address 127.3.247.130:39225)
0806 06:54:36.931125 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:36.931177 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:36.931748 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:36.932042 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:36.932052 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:36.932626 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:36.932855 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:36.932861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.933599 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.933600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:36.934449 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:36.934451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:36.934864 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:36.934865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:36.935024 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:36.935505 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:36.935513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:36.937000 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:36.939114 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:36.939122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:36.939125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:36.939378 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:36.939588 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:36.939589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:36.939589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:36.939736 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:36.940048 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:36.940054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:36.940237 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 06:54:36.940393 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 06:54:36.940582 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260806 06:54:36.941318 22435 heartbeater.cc:347] Connected to a master server at 127.3.247.190:45415
I20260806 06:54:36.941407 22435 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:36.941607 22435 heartbeater.cc:511] Master 127.3.247.190:45415 requested a full tablet report, sending...
I20260806 06:54:36.942049 22110 ts_manager.cc:194] Registered new tserver with Master: 742f4d89e8b240d9ae5b7433649b4af9 (127.3.247.130:34909)
I20260806 06:54:36.942479 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:39225
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:54:36.964244 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:37.045454 22440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:37.045706 22440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:37.045747 22440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:37.047060 22440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:37.047176 22440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:37.047248 22440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:37.049310 22440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:45415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:37.049597 22440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:37.049921 22440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:37.055087 22448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.055130 22446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.055087 22445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.055377 22440 server_base.cc:1034] running on GCE node
I20260806 06:54:37.055611 22440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:37.055876 22440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:37.057049 22440 hybrid_clock.cc:648] HybridClock initialized: now 1785999277057042 us; error 40 us; skew 500 ppm
Aug 06 06:54:37 dist-test-slave-0kls krb5kdc[22065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999277, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:37.059620 22440 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:54:37.060092 22440 webserver.cc:460] Webserver started at http://127.3.247.131:46799/ using document root <none> and password file <none>
I20260806 06:54:37.060338 22440 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:37.060407 22440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:37.060554 22440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:37.061630 22440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "8e0d3d8475ad4d37aed879e7205c93f1"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "4fe2ec1a5814d99e02f1269783765629"
server_key_iv: "500dbf1939777299e2a86ee49a60da5b"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.062013 22440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "8e0d3d8475ad4d37aed879e7205c93f1"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "4fe2ec1a5814d99e02f1269783765629"
server_key_iv: "500dbf1939777299e2a86ee49a60da5b"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.063376 22440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:37.064157 22455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.064350 22440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:37.064445 22440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "8e0d3d8475ad4d37aed879e7205c93f1"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "4fe2ec1a5814d99e02f1269783765629"
server_key_iv: "500dbf1939777299e2a86ee49a60da5b"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.064499 22440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:37.082077 22440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:37.084666 22440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:37.084791 22440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:37.084993 22440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:37.085318 22440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:37.085354 22440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.085377 22440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:37.085433 22440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.091225 22440 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:42791
I20260806 06:54:37.091290 22568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:42791 every 8 connection(s)
I20260806 06:54:37.091639 22440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:54:37 dist-test-slave-0kls krb5kdc[22065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999277, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:37.099123 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22440
I20260806 06:54:37.099237 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:37.099464 4062 external_mini_cluster.cc:1428] Setting key 65c8c630723ef3b428db0cbda95c7c03
I20260806 06:54:37.101764 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.092410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:53047 (local address 127.3.247.190:45415)
0806 06:54:37.092492 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.092495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.092680 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:54:37.093213 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.093214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.093228 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.093288 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.094373 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.094957 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.095971 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.096184 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.096279 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.098468 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.098469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.098470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:37.098478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.100220 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:37.100718 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:37.100720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:37.100720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:37.100790 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:37.101062 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:37.101063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:37.101063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:37.101194 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.101288 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.101581 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.101668 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:54:37.102008 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.092385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:45415 (local address 127.3.247.131:53047)
0806 06:54:37.092559 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.092594 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.093100 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.093359 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.093367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.094064 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:37.094264 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.094269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.095047 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.095048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.095862 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.095864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.096290 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.096291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.096417 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.096906 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:37.096915 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:37.098363 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:37.100304 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:37.100313 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:37.100315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:37.100632 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:37.100862 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:37.100863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:37.100863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:37.100983 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:37.101280 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:37.101287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:37.101441 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.101559 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.101730 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260806 06:54:37.102494 22569 heartbeater.cc:347] Connected to a master server at 127.3.247.190:45415
I20260806 06:54:37.102610 22569 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:37.102804 22569 heartbeater.cc:511] Master 127.3.247.190:45415 requested a full tablet report, sending...
I20260806 06:54:37.103148 22110 ts_manager.cc:194] Registered new tserver with Master: 8e0d3d8475ad4d37aed879e7205c93f1 (127.3.247.131:42791)
I20260806 06:54:37.103531 22110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:53047
I20260806 06:54:37.111511 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:37.119181 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.113129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.3.247.190:45415)
0806 06:54:37.113179 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.113181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.113283 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:54:37.113336 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.113337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.113345 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.113396 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.113987 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.114481 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.115189 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.115382 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.115434 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.116852 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.116853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.116854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:37.116859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.118099 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:37.118467 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:37.118468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:37.118468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:37.118518 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:37.118704 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:37.118705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:37.118705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:37.118827 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.118903 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.119005 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.119052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:37.120906 22110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:37.122527 22110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:37.128867 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.124932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.3.247.131:42791)
0806 06:54:37.125125 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.125129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.125138 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:54:37.125187 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.125189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.125201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.125301 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.125540 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.126383 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.127676 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.128414 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128501 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.128561 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.128739 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.128779 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260806 06:54:37.128856 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.124754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:42791 (local address 127.0.0.1:35774)
0806 06:54:37.124927 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.124966 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.125035 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.125306 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.125309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.125312 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.125386 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.125387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.126491 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.126491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.127580 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.127583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.128534 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.128620 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.128630 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.128719 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.128779 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
I20260806 06:54:37.129108 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.124762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37507 (local address 127.0.0.1:38480)
0806 06:54:37.125231 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.125240 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.125324 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.125561 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.125562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.125564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.125622 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.125624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.126974 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.126975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.127929 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.127931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.128875 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.128948 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.128952 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.128992 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.129033 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":36,"thread_start_us":99,"threads_started":1}
I20260806 06:54:37.129294 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.124767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:34909 (local address 127.0.0.1:41454)
0806 06:54:37.125399 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.125409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.125455 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.125720 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.125721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.125723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.125758 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.125760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.127008 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.127009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.128143 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.128145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.129051 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.129052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.129107 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.129110 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.129157 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.129231 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":21,"thread_start_us":79,"threads_started":1}
I20260806 06:54:37.129434 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.124998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.3.247.129:37507)
0806 06:54:37.125192 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.125195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.125314 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:54:37.125391 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.125393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.125407 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.125478 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.125703 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.126857 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128030 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.128722 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128792 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.129285 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.129312 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.129360 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260806 06:54:37.129747 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.125013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.3.247.130:34909)
0806 06:54:37.125183 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.125187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.125516 (+ 329us) server_negotiation.cc:407] Connection header received
0806 06:54:37.125561 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.125563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.125580 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.125647 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.125882 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.126906 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128229 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.128906 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.128969 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.129020 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.129390 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.129434 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260806 06:54:37.130111 22503 tablet_service.cc:1467] Processing CreateTablet for tablet 393c709f800948ed8459d89c907044b9 (DEFAULT_TABLE table=test-table [id=5c269928c4064c6b9743fddbd28727fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.130185 22235 tablet_service.cc:1467] Processing CreateTablet for tablet 393c709f800948ed8459d89c907044b9 (DEFAULT_TABLE table=test-table [id=5c269928c4064c6b9743fddbd28727fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.130366 22369 tablet_service.cc:1467] Processing CreateTablet for tablet 393c709f800948ed8459d89c907044b9 (DEFAULT_TABLE table=test-table [id=5c269928c4064c6b9743fddbd28727fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.130537 22503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393c709f800948ed8459d89c907044b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.130546 22235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393c709f800948ed8459d89c907044b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.130646 22369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393c709f800948ed8459d89c907044b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.133015 22586 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Bootstrap starting.
I20260806 06:54:37.133821 22586 tablet_bootstrap.cc:654] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.133957 22587 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Bootstrap starting.
I20260806 06:54:37.134268 22588 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: Bootstrap starting.
I20260806 06:54:37.134618 22586 log.cc:824] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.134678 22587 tablet_bootstrap.cc:654] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.134930 22588 tablet_bootstrap.cc:654] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.135232 22587 log.cc:824] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.135495 22586 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: No bootstrap required, opened a new log
I20260806 06:54:37.135528 22588 log.cc:824] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.135567 22586 ts_tablet_manager.cc:1397] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:54:37.135968 22587 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: No bootstrap required, opened a new log
I20260806 06:54:37.136037 22587 ts_tablet_manager.cc:1397] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:37.136433 22588 tablet_bootstrap.cc:492] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: No bootstrap required, opened a new log
I20260806 06:54:37.136500 22588 ts_tablet_manager.cc:1397] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:37.137554 22586 raft_consensus.cc:348] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.137744 22586 raft_consensus.cc:374] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.137809 22586 raft_consensus.cc:729] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0d3d8475ad4d37aed879e7205c93f1, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.137946 22586 consensus_queue.cc:260] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.137926 22588 raft_consensus.cc:348] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.138106 22588 raft_consensus.cc:374] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.138149 22588 raft_consensus.cc:729] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70425d0d222e4e30b4b88b02bb8042e2, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.138090 22587 raft_consensus.cc:348] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.138231 22586 ts_tablet_manager.cc:1428] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:54:37.138258 22587 raft_consensus.cc:374] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.138290 22587 raft_consensus.cc:729] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 742f4d89e8b240d9ae5b7433649b4af9, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.138276 22588 consensus_queue.cc:260] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.138341 22569 heartbeater.cc:503] Master 127.3.247.190:45415 was elected leader, sending a full tablet report...
I20260806 06:54:37.138427 22587 consensus_queue.cc:260] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.138540 22588 ts_tablet_manager.cc:1428] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:54:37.138648 22301 heartbeater.cc:503] Master 127.3.247.190:45415 was elected leader, sending a full tablet report...
I20260806 06:54:37.138692 22587 ts_tablet_manager.cc:1428] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:54:37.138801 22435 heartbeater.cc:503] Master 127.3.247.190:45415 was elected leader, sending a full tablet report...
W20260806 06:54:37.180990 22436 tablet.cc:2369] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:37.217259 22593 raft_consensus.cc:482] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:37.217495 22593 raft_consensus.cc:504] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.217933 22593 leader_election.cc:288] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e0d3d8475ad4d37aed879e7205c93f1 (127.3.247.131:42791), 742f4d89e8b240d9ae5b7433649b4af9 (127.3.247.130:34909)
I20260806 06:54:37.221688 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.218179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:38663 (local address 127.3.247.131:42791)
0806 06:54:37.218263 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.218265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.218276 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:54:37.218305 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.218306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.218317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.218376 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.218574 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.219354 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.220747 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.221328 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.221411 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.221420 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.221573 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.221610 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:54:37.221693 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.218121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:42791 (local address 127.3.247.129:38663)
0806 06:54:37.218168 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.218181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.218225 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.218413 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.218414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.218417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.218492 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.218495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.219435 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.219436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.220642 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.220645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.221411 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.221412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.221493 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.221496 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.221566 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.221610 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:54:37.221720 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.218242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:43977 (local address 127.3.247.130:34909)
0806 06:54:37.218308 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.218310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.218579 (+ 269us) server_negotiation.cc:407] Connection header received
0806 06:54:37.218608 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.218609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.218619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.218668 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.218808 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.219505 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.220797 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.221331 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.221415 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.221423 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.221590 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.221629 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:54:37.221722 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.218129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:34909 (local address 127.3.247.129:43977)
0806 06:54:37.218465 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.218476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.218543 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.218673 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.218674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.218675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.218739 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.218740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.219571 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.219572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.220661 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.220663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.221428 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.221429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.221507 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.221510 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.221571 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.221622 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":151,"thread_start_us":110,"threads_started":1}
I20260806 06:54:37.222090 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "393c709f800948ed8459d89c907044b9" candidate_uuid: "70425d0d222e4e30b4b88b02bb8042e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "742f4d89e8b240d9ae5b7433649b4af9" is_pre_election: true
I20260806 06:54:37.222105 22523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "393c709f800948ed8459d89c907044b9" candidate_uuid: "70425d0d222e4e30b4b88b02bb8042e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" is_pre_election: true
I20260806 06:54:37.222266 22389 raft_consensus.cc:2454] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70425d0d222e4e30b4b88b02bb8042e2 in term 0.
I20260806 06:54:37.222268 22523 raft_consensus.cc:2454] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70425d0d222e4e30b4b88b02bb8042e2 in term 0.
I20260806 06:54:37.222506 22191 leader_election.cc:302] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 70425d0d222e4e30b4b88b02bb8042e2, 742f4d89e8b240d9ae5b7433649b4af9, 8e0d3d8475ad4d37aed879e7205c93f1; no voters:
I20260806 06:54:37.222645 22593 raft_consensus.cc:2781] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:37.222718 22593 raft_consensus.cc:482] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:37.222735 22593 raft_consensus.cc:3037] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.223421 22593 raft_consensus.cc:504] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.223567 22593 leader_election.cc:288] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [CANDIDATE]: Term 1 election: Requested vote from peers 8e0d3d8475ad4d37aed879e7205c93f1 (127.3.247.131:42791), 742f4d89e8b240d9ae5b7433649b4af9 (127.3.247.130:34909)
I20260806 06:54:37.223734 22523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "393c709f800948ed8459d89c907044b9" candidate_uuid: "70425d0d222e4e30b4b88b02bb8042e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e0d3d8475ad4d37aed879e7205c93f1"
I20260806 06:54:37.223759 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "393c709f800948ed8459d89c907044b9" candidate_uuid: "70425d0d222e4e30b4b88b02bb8042e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "742f4d89e8b240d9ae5b7433649b4af9"
I20260806 06:54:37.223814 22523 raft_consensus.cc:3037] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.223824 22389 raft_consensus.cc:3037] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.224681 22523 raft_consensus.cc:2454] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70425d0d222e4e30b4b88b02bb8042e2 in term 1.
I20260806 06:54:37.224681 22389 raft_consensus.cc:2454] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70425d0d222e4e30b4b88b02bb8042e2 in term 1.
I20260806 06:54:37.224879 22191 leader_election.cc:302] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70425d0d222e4e30b4b88b02bb8042e2, 742f4d89e8b240d9ae5b7433649b4af9; no voters:
I20260806 06:54:37.225009 22593 raft_consensus.cc:2781] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:37.225178 22593 raft_consensus.cc:686] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 1 LEADER]: Becoming Leader. State: Replica: 70425d0d222e4e30b4b88b02bb8042e2, State: Running, Role: LEADER
I20260806 06:54:37.225304 22593 consensus_queue.cc:237] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } }
I20260806 06:54:37.226198 22108 catalog_manager.cc:5495] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70425d0d222e4e30b4b88b02bb8042e2 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "70425d0d222e4e30b4b88b02bb8042e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70425d0d222e4e30b4b88b02bb8042e2" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37507 } health_report { overall_health: HEALTHY } } }
I20260806 06:54:37.248838 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.246644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.3.247.129:37507)
0806 06:54:37.246705 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.246707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.246717 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:37.246754 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.246755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.246776 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.246835 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:37.246954 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.247426 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.248131 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.248323 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.248371 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.248500 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:37.248583 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:37.248651 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.248694 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.248703 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:54:37.250669 22523 raft_consensus.cc:1261] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 1 FOLLOWER]: Refusing update from remote peer 70425d0d222e4e30b4b88b02bb8042e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:37.250669 22389 raft_consensus.cc:1261] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 1 FOLLOWER]: Refusing update from remote peer 70425d0d222e4e30b4b88b02bb8042e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:37.251047 22593 consensus_queue.cc:1035] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e0d3d8475ad4d37aed879e7205c93f1" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:37.251221 22596 consensus_queue.cc:1035] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "742f4d89e8b240d9ae5b7433649b4af9" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:37.253556 22599 mvcc.cc:205] Tried to move back new op lower bound from 7315453039616065536 to 7315453039515459584. Current Snapshot: MvccSnapshot[applied={T|T < 7315453039616065536}]
I20260806 06:54:37.254292 22600 mvcc.cc:205] Tried to move back new op lower bound from 7315453039616065536 to 7315453039515459584. Current Snapshot: MvccSnapshot[applied={T|T < 7315453039616065536}]
I20260806 06:54:37.254743 22601 mvcc.cc:205] Tried to move back new op lower bound from 7315453039616065536 to 7315453039515459584. Current Snapshot: MvccSnapshot[applied={T|T < 7315453039616065536 or (T in {7315453039616065536})}]
I20260806 06:54:37.256176 22110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:37.256260 22110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:37.256981 22110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edc3edc0e61845df9cf4fa4b4d315a91: Sending DeleteTablet for 3 replicas of tablet 393c709f800948ed8459d89c907044b9
I20260806 06:54:37.257323 22369 tablet_service.cc:1514] Processing DeleteTablet for tablet 393c709f800948ed8459d89c907044b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:41454
I20260806 06:54:37.257453 22503 tablet_service.cc:1514] Processing DeleteTablet for tablet 393c709f800948ed8459d89c907044b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:35774
I20260806 06:54:37.257553 22609 tablet_replica.cc:330] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: stopping tablet replica
I20260806 06:54:37.257547 22235 tablet_service.cc:1514] Processing DeleteTablet for tablet 393c709f800948ed8459d89c907044b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:38480
I20260806 06:54:37.257647 22609 raft_consensus.cc:2229] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:37.257766 22609 raft_consensus.cc:2258] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:37.257915 22611 tablet_replica.cc:330] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2: stopping tablet replica
I20260806 06:54:37.258046 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22172
I20260806 06:54:37.258059 22611 raft_consensus.cc:2229] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:54:37.258229 22611 raft_consensus.cc:2258] T 393c709f800948ed8459d89c907044b9 P 70425d0d222e4e30b4b88b02bb8042e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:37.258401 22610 tablet_replica.cc:330] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: stopping tablet replica
I20260806 06:54:37.258508 22610 raft_consensus.cc:2229] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:37.258630 22610 raft_consensus.cc:2258] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:37.258700 22609 ts_tablet_manager.cc:1905] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:37.259092 22610 ts_tablet_manager.cc:1905] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:37.260545 22610 ts_tablet_manager.cc:1918] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:37.260632 22610 log.cc:1196] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/393c709f800948ed8459d89c907044b9
I20260806 06:54:37.260759 22609 ts_tablet_manager.cc:1918] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:37.260833 22609 log.cc:1196] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/393c709f800948ed8459d89c907044b9
I20260806 06:54:37.260931 22610 ts_tablet_manager.cc:1939] T 393c709f800948ed8459d89c907044b9 P 8e0d3d8475ad4d37aed879e7205c93f1: Deleting consensus metadata
I20260806 06:54:37.261096 22609 ts_tablet_manager.cc:1939] T 393c709f800948ed8459d89c907044b9 P 742f4d89e8b240d9ae5b7433649b4af9: Deleting consensus metadata
I20260806 06:54:37.261385 22094 catalog_manager.cc:4841] TS 8e0d3d8475ad4d37aed879e7205c93f1 (127.3.247.131:42791): tablet 393c709f800948ed8459d89c907044b9 (table test-table [id=5c269928c4064c6b9743fddbd28727fa]) successfully deleted
I20260806 06:54:37.261554 22097 catalog_manager.cc:4841] TS 742f4d89e8b240d9ae5b7433649b4af9 (127.3.247.130:34909): tablet 393c709f800948ed8459d89c907044b9 (table test-table [id=5c269928c4064c6b9743fddbd28727fa]) successfully deleted
W20260806 06:54:37.264894 22095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:37507 (error 108)
I20260806 06:54:37.265089 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22306
I20260806 06:54:37.266219 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.266089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37507 (local address 127.0.0.1:38496)
0806 06:54:37.266149 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.266189 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 06:54:37.266377 22095 catalog_manager.cc:4584] TS 70425d0d222e4e30b4b88b02bb8042e2 (127.3.247.129:37507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 393c709f800948ed8459d89c907044b9: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37507: connect: Connection refused (error 111)
I20260806 06:54:37.270507 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22440
I20260806 06:54:37.275043 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22078
2026-08-06T06:54:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1985 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T06:54:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:37Z Disabled control of system clock
I20260806 06:54:37.302719 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:44843
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:44843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:37.383091 22616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:37.383315 22616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:37.383337 22616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:37.384528 22616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:37.384572 22616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:37.384585 22616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:37.384598 22616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:37.384658 22616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:37.386060 22616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:44843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:44843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:37.386323 22616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:37.386652 22616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:54:37.391691 22616 server_base.cc:1034] running on GCE node
W20260806 06:54:37.391644 22622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.391667 22621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.391641 22624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.392168 22616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:37.392473 22616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:37.393641 22616 hybrid_clock.cc:648] HybridClock initialized: now 1785999277393633 us; error 35 us; skew 500 ppm
I20260806 06:54:37.399811 22616 webserver.cc:460] Webserver started at http://127.3.247.190:35321/ using document root <none> and password file <none>
I20260806 06:54:37.400131 22616 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:37.400214 22616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:37.400384 22616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:37.401530 22616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "5b16bb0a46b845948a551ad3811946d6"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "71469ec6e08dde9e8ab6ee39f6dce1b2"
server_key_iv: "bc923c2d177b8247f1e2c299db33c119"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.401968 22616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "5b16bb0a46b845948a551ad3811946d6"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "71469ec6e08dde9e8ab6ee39f6dce1b2"
server_key_iv: "bc923c2d177b8247f1e2c299db33c119"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.403314 22616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:37.404064 22630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.404242 22616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:37.404346 22616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "5b16bb0a46b845948a551ad3811946d6"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "71469ec6e08dde9e8ab6ee39f6dce1b2"
server_key_iv: "bc923c2d177b8247f1e2c299db33c119"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.404430 22616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:37.418990 22616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:37.419334 22616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:37.419484 22616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:37.423455 22616 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:44843
I20260806 06:54:37.423493 22682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:44843 every 8 connection(s)
I20260806 06:54:37.423856 22616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:37.424903 22683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.427122 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22616
I20260806 06:54:37.427261 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:37.427336 22683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: Bootstrap starting.
I20260806 06:54:37.427506 4062 external_mini_cluster.cc:1428] Setting key 5b6cb4eccaa7f4b4a09cc413dcf6cb98
I20260806 06:54:37.428066 22683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.428385 22683 log.cc:824] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.429111 22683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: No bootstrap required, opened a new log
I20260806 06:54:37.431304 22683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } }
I20260806 06:54:37.431476 22683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.431509 22683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b16bb0a46b845948a551ad3811946d6, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.431666 22683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } }
I20260806 06:54:37.431744 22683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:37.431792 22683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:37.431845 22683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.432525 22683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } }
I20260806 06:54:37.432657 22683 leader_election.cc:302] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b16bb0a46b845948a551ad3811946d6; no voters:
I20260806 06:54:37.432852 22683 leader_election.cc:288] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:37.432880 22688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:37.433051 22688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [term 1 LEADER]: Becoming Leader. State: Replica: 5b16bb0a46b845948a551ad3811946d6, State: Running, Role: LEADER
I20260806 06:54:37.433145 22683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:37.433221 22688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } }
I20260806 06:54:37.433557 22688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b16bb0a46b845948a551ad3811946d6. Latest consensus state: current_term: 1 leader_uuid: "5b16bb0a46b845948a551ad3811946d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } } }
I20260806 06:54:37.433637 22688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:37.433838 22689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b16bb0a46b845948a551ad3811946d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16bb0a46b845948a551ad3811946d6" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 44843 } } }
I20260806 06:54:37.433934 22689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:37.434042 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.428049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.3.247.190:44843)
0806 06:54:37.428306 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.428314 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.428343 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:54:37.428953 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.428962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.429033 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.429155 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.429431 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.430406 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.431597 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.432136 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.432234 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.433060 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.433065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.433065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:37.433308 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.433354 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:37.433369 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.433425 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.433689 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.433775 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260806 06:54:37.434667 22695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:37.435281 22695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:37.436735 22695 catalog_manager.cc:1347] Generated new cluster ID: e31a0a6bbc7c4ff88facc53b7f301d39
I20260806 06:54:37.436802 22695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:37.442117 22695 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:37.442813 22695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:37.460161 22695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: Generated new TSK 0
I20260806 06:54:37.460410 22695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:37.462433 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--builtin_ntp_servers=127.3.247.148:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:37.544664 22707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:37.544941 22707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:37.545011 22707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:37.546491 22707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:37.546559 22707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:37.546604 22707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:37.547999 22707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:37.548272 22707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:37.548589 22707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:37.554473 22715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.554471 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.554502 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.554764 22707 server_base.cc:1034] running on GCE node
I20260806 06:54:37.554988 22707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:37.555236 22707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:37.556444 22707 hybrid_clock.cc:648] HybridClock initialized: now 1785999277556421 us; error 46 us; skew 500 ppm
I20260806 06:54:37.557993 22707 webserver.cc:460] Webserver started at http://127.3.247.129:42585/ using document root <none> and password file <none>
I20260806 06:54:37.558390 22707 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:37.558461 22707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:37.558602 22707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:37.560138 22707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "3d726e07fede4598a6dab7210f97b0ff"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "59904ade64f48ec5be9ca1a697cb1f43"
server_key_iv: "83a310ee2693464297ac992d8f64697e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.560596 22707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "3d726e07fede4598a6dab7210f97b0ff"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "59904ade64f48ec5be9ca1a697cb1f43"
server_key_iv: "83a310ee2693464297ac992d8f64697e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.562352 22707 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 06:54:37.563202 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.563387 22707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:37.563481 22707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "3d726e07fede4598a6dab7210f97b0ff"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "59904ade64f48ec5be9ca1a697cb1f43"
server_key_iv: "83a310ee2693464297ac992d8f64697e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.563550 22707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:37.584971 22707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:37.585280 22707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:37.585389 22707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:37.585587 22707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:37.586040 22707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:37.586079 22707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.586103 22707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:37.586153 22707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.591964 22707 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:43085
I20260806 06:54:37.592006 22834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:43085 every 8 connection(s)
I20260806 06:54:37.592337 22707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:54:37.597620 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22707
I20260806 06:54:37.597724 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:37.597896 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.593041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:57677 (local address 127.3.247.190:44843)
0806 06:54:37.593083 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.593085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.593344 (+ 259us) server_negotiation.cc:407] Connection header received
0806 06:54:37.593941 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.593942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.593954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.594025 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.594906 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.595418 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.596387 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.596589 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.596669 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.597541 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.597542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.597543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:37.597549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.597568 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:37.597573 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.597641 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.597769 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.597817 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:37.597980 4062 external_mini_cluster.cc:1428] Setting key 73ba60f44edea4ef94b68b8cbde13569
I20260806 06:54:37.598114 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.593058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44843 (local address 127.3.247.129:57677)
0806 06:54:37.593223 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.593267 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.593751 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.594086 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.594089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.594097 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:37.594602 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.594606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.595510 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.595511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.596288 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.596290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.596710 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.596711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.596829 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.597382 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:37.597382 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:37.597411 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:37.597434 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:37.597646 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:37.597650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:37.597652 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.597755 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.597890 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260806 06:54:37.598618 22835 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44843
I20260806 06:54:37.598773 22835 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:37.598984 22835 heartbeater.cc:511] Master 127.3.247.190:44843 requested a full tablet report, sending...
I20260806 06:54:37.599000 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--builtin_ntp_servers=127.3.247.148:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:54:37.599537 22647 ts_manager.cc:194] Registered new tserver with Master: 3d726e07fede4598a6dab7210f97b0ff (127.3.247.129:43085)
I20260806 06:54:37.600455 22647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.129:57677
W20260806 06:54:37.682041 22838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:37.682247 22838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:37.682269 22838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:37.683449 22838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:37.683487 22838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:37.683542 22838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:37.684875 22838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:37.685117 22838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:37.685418 22838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:37.690349 22846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.690418 22838 server_base.cc:1034] running on GCE node
W20260806 06:54:37.690366 22844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.690367 22843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.690822 22838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:37.691038 22838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:37.692209 22838 hybrid_clock.cc:648] HybridClock initialized: now 1785999277692200 us; error 39 us; skew 500 ppm
I20260806 06:54:37.693454 22838 webserver.cc:460] Webserver started at http://127.3.247.130:45683/ using document root <none> and password file <none>
I20260806 06:54:37.693694 22838 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:37.693773 22838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:37.693949 22838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:37.694993 22838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "c8acb30225144d2fb555a3015b94b8d5"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "538eb91e669eeb0472805bee24c8f635"
server_key_iv: "6a1c97ec9c003fe15ede6a07cefe14e3"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.695288 22838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "c8acb30225144d2fb555a3015b94b8d5"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "538eb91e669eeb0472805bee24c8f635"
server_key_iv: "6a1c97ec9c003fe15ede6a07cefe14e3"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.696516 22838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:54:37.697182 22852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.697340 22838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:37.697407 22838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "c8acb30225144d2fb555a3015b94b8d5"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "538eb91e669eeb0472805bee24c8f635"
server_key_iv: "6a1c97ec9c003fe15ede6a07cefe14e3"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.697458 22838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:37.716336 22838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:37.716604 22838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:37.716706 22838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:37.716894 22838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:37.717162 22838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:37.717192 22838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.717216 22838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:37.717227 22838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.723271 22838 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:40699
I20260806 06:54:37.723309 22965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:40699 every 8 connection(s)
I20260806 06:54:37.723670 22838 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:54:37.724320 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22838
I20260806 06:54:37.724437 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:37.724694 4062 external_mini_cluster.cc:1428] Setting key 79a493344cb4c12e58aa71c40ee2dc1f
I20260806 06:54:37.725754 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--builtin_ntp_servers=127.3.247.148:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:54:37.729825 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.724485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:59395 (local address 127.3.247.190:44843)
0806 06:54:37.724559 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.724561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.724843 (+ 282us) server_negotiation.cc:407] Connection header received
0806 06:54:37.725537 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.725538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.725547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.725590 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.726535 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.727121 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.728148 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.728496 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.728570 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.729481 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.729483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.729483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:37.729489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.729509 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:37.729514 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.729579 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.729648 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.729680 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:54:37.730010 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.724472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44843 (local address 127.3.247.130:59395)
0806 06:54:37.724692 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.724740 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.725437 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.725662 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.725666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.725677 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:37.726433 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.726441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.727262 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.727263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.728054 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.728056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.728581 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.728581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.728693 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.729320 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:37.729320 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:37.729350 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:37.729382 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:37.729584 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:37.729588 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:37.729590 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.729642 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.729726 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260806 06:54:37.730530 22966 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44843
I20260806 06:54:37.730635 22966 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:37.730819 22966 heartbeater.cc:511] Master 127.3.247.190:44843 requested a full tablet report, sending...
I20260806 06:54:37.731236 22647 ts_manager.cc:194] Registered new tserver with Master: c8acb30225144d2fb555a3015b94b8d5 (127.3.247.130:40699)
I20260806 06:54:37.731815 22647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.130:59395
W20260806 06:54:37.809000 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:37.809226 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:37.809247 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:37.810384 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:37.810420 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:37.810446 22969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:37.811751 22969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:44843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:37.811954 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:37.812206 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:37.817420 22975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:37.817420 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.817525 22969 server_base.cc:1034] running on GCE node
W20260806 06:54:37.817471 22974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:37.817917 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:37.818177 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:37.819478 22969 hybrid_clock.cc:648] HybridClock initialized: now 1785999277819453 us; error 33 us; skew 500 ppm
I20260806 06:54:37.820849 22969 webserver.cc:460] Webserver started at http://127.3.247.131:36859/ using document root <none> and password file <none>
I20260806 06:54:37.821107 22969 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:37.821180 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:37.821337 22969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:37.822533 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "10fff915f57643a6b5b52e8e4c0a492e"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "423e1b340bb5a40d98255c7e4c709053"
server_key_iv: "3b3e1e3de167d89621bb70b18e548c3e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.822901 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "10fff915f57643a6b5b52e8e4c0a492e"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "423e1b340bb5a40d98255c7e4c709053"
server_key_iv: "3b3e1e3de167d89621bb70b18e548c3e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.824193 22969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:54:37.824918 22983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.825086 22969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:37.825186 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "10fff915f57643a6b5b52e8e4c0a492e"
format_stamp: "Formatted at 2026-08-06 06:54:37 on dist-test-slave-0kls"
server_key: "423e1b340bb5a40d98255c7e4c709053"
server_key_iv: "3b3e1e3de167d89621bb70b18e548c3e"
server_key_version: "encryptionkey@0"
I20260806 06:54:37.825278 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:37.840399 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:37.840734 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:37.840893 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:37.841166 22969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:37.841509 22969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:37.841569 22969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.841611 22969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:37.841655 22969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:37.847510 22969 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:42755
I20260806 06:54:37.847553 23096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:42755 every 8 connection(s)
I20260806 06:54:37.847898 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:37.851440 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 22969
I20260806 06:54:37.851547 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:37.851745 4062 external_mini_cluster.cc:1428] Setting key 6814311e219f8e27b20f7654665aba79
I20260806 06:54:37.853665 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.848680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:60233 (local address 127.3.247.190:44843)
0806 06:54:37.848768 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.848771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.848993 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:54:37.849486 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.849487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.849497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.849552 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.850305 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.850785 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.852296 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.852576 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.852647 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.853433 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.853434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.853434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:37.853438 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.853452 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:37.853455 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.853500 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.853586 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.853606 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:54:37.853917 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.848649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:44843 (local address 127.3.247.131:60233)
0806 06:54:37.848864 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.848899 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.849373 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.849617 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.849620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.849628 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:37.850204 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.850210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.850866 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.850867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.852208 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.852211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.852691 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.852692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.852833 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.853303 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:37.853304 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:37.853333 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:37.853356 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:37.853520 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:37.853523 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:37.853526 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.853575 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.853669 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260806 06:54:37.854405 23097 heartbeater.cc:347] Connected to a master server at 127.3.247.190:44843
I20260806 06:54:37.854496 23097 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:37.854636 23097 heartbeater.cc:511] Master 127.3.247.190:44843 requested a full tablet report, sending...
I20260806 06:54:37.854977 22647 ts_manager.cc:194] Registered new tserver with Master: 10fff915f57643a6b5b52e8e4c0a492e (127.3.247.131:42755)
I20260806 06:54:37.855388 22647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.131:60233
I20260806 06:54:37.863469 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:37.868126 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.864857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.3.247.190:44843)
0806 06:54:37.864928 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.864931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.865015 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:54:37.865091 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.865092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.865102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.865168 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:37.865347 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.865974 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.866746 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.867028 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.867100 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.867804 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:37.867805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:37.867806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:37.867810 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:37.867824 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:37.867828 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:37.867943 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.868029 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.868040 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:54:37.869876 22647 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:37.871433 22647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:37.878224 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.874149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:42755 (local address 127.0.0.1:57528)
0806 06:54:37.874304 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.874329 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.874378 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.874587 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.874590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.874593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.874684 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.874686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.876017 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.876018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.877064 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.877066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.877900 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.877901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.877976 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.877990 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.878060 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.878116 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260806 06:54:37.878321 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.873934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.3.247.129:43085)
0806 06:54:37.874478 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.874483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.874494 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:54:37.874536 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.874539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.874556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.874879 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.874948 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.876012 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.877129 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.877930 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.878045 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.878120 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.878162 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.878202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":88,"threads_started":1}
I20260806 06:54:37.878523 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.873966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.3.247.130:40699)
0806 06:54:37.874175 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.874180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.874381 (+ 201us) server_negotiation.cc:407] Connection header received
0806 06:54:37.874750 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.874753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.874769 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.874846 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.874951 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.875638 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.877182 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.877933 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.878014 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.878103 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.878376 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.878428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260806 06:54:37.878583 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.874218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.3.247.131:42755)
0806 06:54:37.874408 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.874413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.874423 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:37.874465 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.874467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.874484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.874566 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.874779 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.875924 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.877162 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.877828 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.877900 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.877996 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.878471 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.878510 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260806 06:54:37.878239 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.873827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43085 (local address 127.0.0.1:58478)
0806 06:54:37.874010 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.874054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.874122 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.874761 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.874762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.874764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.874795 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.874796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.876094 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.876095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.877035 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.877038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.878052 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.878053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.878105 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.878107 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.878148 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.878178 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260806 06:54:37.879047 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.873874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40699 (local address 127.0.0.1:46580)
0806 06:54:37.874299 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.874311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.874647 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.874845 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.874846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.874848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.874894 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.874895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.875731 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.875732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.877061 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.877063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.878142 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.878144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.878216 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.878221 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.878281 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.878323 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
I20260806 06:54:37.879318 23031 tablet_service.cc:1467] Processing CreateTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 (DEFAULT_TABLE table=test-table [id=3099d27c7aab4ccb814902c3a66530f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.879541 22769 tablet_service.cc:1467] Processing CreateTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 (DEFAULT_TABLE table=test-table [id=3099d27c7aab4ccb814902c3a66530f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.879717 23031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74058af1acb04fb88c5c48889d2cc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.879877 22769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74058af1acb04fb88c5c48889d2cc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.883487 23111 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: Bootstrap starting.
I20260806 06:54:37.884123 23111 tablet_bootstrap.cc:654] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.884447 23111 log.cc:824] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.884394 22900 tablet_service.cc:1467] Processing CreateTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 (DEFAULT_TABLE table=test-table [id=3099d27c7aab4ccb814902c3a66530f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:37.884774 22900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74058af1acb04fb88c5c48889d2cc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:37.884804 23112 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Bootstrap starting.
I20260806 06:54:37.885246 23111 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: No bootstrap required, opened a new log
I20260806 06:54:37.885320 23111 ts_tablet_manager.cc:1397] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:37.885442 23112 tablet_bootstrap.cc:654] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.886026 23112 log.cc:824] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.886754 23112 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: No bootstrap required, opened a new log
I20260806 06:54:37.886822 23112 ts_tablet_manager.cc:1397] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:37.887907 23111 raft_consensus.cc:348] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.888092 23111 raft_consensus.cc:374] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.888134 23111 raft_consensus.cc:729] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d726e07fede4598a6dab7210f97b0ff, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.888249 23111 consensus_queue.cc:260] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.888526 23111 ts_tablet_manager.cc:1428] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:54:37.888531 23115 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: Bootstrap starting.
I20260806 06:54:37.888684 22835 heartbeater.cc:503] Master 127.3.247.190:44843 was elected leader, sending a full tablet report...
I20260806 06:54:37.888988 23112 raft_consensus.cc:348] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.889168 23112 raft_consensus.cc:374] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.889183 23115 tablet_bootstrap.cc:654] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:37.889204 23112 raft_consensus.cc:729] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fff915f57643a6b5b52e8e4c0a492e, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.889328 23112 consensus_queue.cc:260] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.889518 23115 log.cc:824] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:37.889575 23112 ts_tablet_manager.cc:1428] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:54:37.889708 23097 heartbeater.cc:503] Master 127.3.247.190:44843 was elected leader, sending a full tablet report...
I20260806 06:54:37.890471 23115 tablet_bootstrap.cc:492] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: No bootstrap required, opened a new log
I20260806 06:54:37.890540 23115 ts_tablet_manager.cc:1397] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:37.892283 23115 raft_consensus.cc:348] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.892405 23115 raft_consensus.cc:374] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:37.892426 23115 raft_consensus.cc:729] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8acb30225144d2fb555a3015b94b8d5, State: Initialized, Role: FOLLOWER
I20260806 06:54:37.892503 23115 consensus_queue.cc:260] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.892679 23115 ts_tablet_manager.cc:1428] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:54:37.892761 22966 heartbeater.cc:503] Master 127.3.247.190:44843 was elected leader, sending a full tablet report...
I20260806 06:54:37.899672 23117 raft_consensus.cc:482] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:37.899827 23117 raft_consensus.cc:504] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.900250 23117 leader_election.cc:288] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d726e07fede4598a6dab7210f97b0ff (127.3.247.129:43085), c8acb30225144d2fb555a3015b94b8d5 (127.3.247.130:40699)
I20260806 06:54:37.904238 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.900491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:40625 (local address 127.3.247.129:43085)
0806 06:54:37.900580 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.900582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.900592 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:37.900618 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.900619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.900630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.900717 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.900942 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.901729 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.903284 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.903872 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.903951 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.903960 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.904109 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.904143 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 06:54:37.904235 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.900420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43085 (local address 127.3.247.131:40625)
0806 06:54:37.900475 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.900487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.900528 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.900734 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.900736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.900737 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.900858 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.900860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.901836 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.901836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.903170 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.903174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.903961 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.903962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.904031 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.904036 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.904093 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.904140 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":61}
I20260806 06:54:37.904350 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.900498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:53523 (local address 127.3.247.130:40699)
0806 06:54:37.900547 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.900549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.900698 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:54:37.900764 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.900765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.900773 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.900835 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:37.900987 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.901728 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.903290 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.903867 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.903951 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.903960 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.904211 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.904256 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:54:37.904596 23120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.900465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40699 (local address 127.3.247.131:53523)
0806 06:54:37.900611 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.900621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:37.900705 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:37.900849 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:37.900850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:37.900851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:37.900921 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.900922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.901824 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.901825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.903170 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:37.903174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.904075 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.904076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:37.904138 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.904140 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:37.904192 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:54:37.904224 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":40,"thread_start_us":103,"threads_started":1}
I20260806 06:54:37.904708 22789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74058af1acb04fb88c5c48889d2cc0c0" candidate_uuid: "10fff915f57643a6b5b52e8e4c0a492e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d726e07fede4598a6dab7210f97b0ff" is_pre_election: true
I20260806 06:54:37.904899 22789 raft_consensus.cc:2454] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fff915f57643a6b5b52e8e4c0a492e in term 0.
I20260806 06:54:37.904924 22920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74058af1acb04fb88c5c48889d2cc0c0" candidate_uuid: "10fff915f57643a6b5b52e8e4c0a492e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8acb30225144d2fb555a3015b94b8d5" is_pre_election: true
I20260806 06:54:37.905067 22920 raft_consensus.cc:2454] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fff915f57643a6b5b52e8e4c0a492e in term 0.
I20260806 06:54:37.905119 22986 leader_election.cc:302] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fff915f57643a6b5b52e8e4c0a492e, 3d726e07fede4598a6dab7210f97b0ff; no voters:
I20260806 06:54:37.905254 23117 raft_consensus.cc:2781] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:37.905332 23117 raft_consensus.cc:482] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:37.905356 23117 raft_consensus.cc:3037] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.906140 23117 raft_consensus.cc:504] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.906296 23117 leader_election.cc:288] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [CANDIDATE]: Term 1 election: Requested vote from peers 3d726e07fede4598a6dab7210f97b0ff (127.3.247.129:43085), c8acb30225144d2fb555a3015b94b8d5 (127.3.247.130:40699)
I20260806 06:54:37.906440 22920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74058af1acb04fb88c5c48889d2cc0c0" candidate_uuid: "10fff915f57643a6b5b52e8e4c0a492e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8acb30225144d2fb555a3015b94b8d5"
I20260806 06:54:37.906540 22920 raft_consensus.cc:3037] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.907346 22920 raft_consensus.cc:2454] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fff915f57643a6b5b52e8e4c0a492e in term 1.
I20260806 06:54:37.907488 22789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74058af1acb04fb88c5c48889d2cc0c0" candidate_uuid: "10fff915f57643a6b5b52e8e4c0a492e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d726e07fede4598a6dab7210f97b0ff"
I20260806 06:54:37.907521 22987 leader_election.cc:302] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fff915f57643a6b5b52e8e4c0a492e, c8acb30225144d2fb555a3015b94b8d5; no voters:
I20260806 06:54:37.907579 22789 raft_consensus.cc:3037] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:37.907612 23117 raft_consensus.cc:2781] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:37.907853 23117 raft_consensus.cc:686] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 1 LEADER]: Becoming Leader. State: Replica: 10fff915f57643a6b5b52e8e4c0a492e, State: Running, Role: LEADER
I20260806 06:54:37.907961 23117 consensus_queue.cc:237] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } }
I20260806 06:54:37.908504 22789 raft_consensus.cc:2454] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fff915f57643a6b5b52e8e4c0a492e in term 1.
I20260806 06:54:37.908819 22646 catalog_manager.cc:5495] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e reported cstate change: term changed from 0 to 1, leader changed from <none> to 10fff915f57643a6b5b52e8e4c0a492e (127.3.247.131). New cstate: current_term: 1 leader_uuid: "10fff915f57643a6b5b52e8e4c0a492e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10fff915f57643a6b5b52e8e4c0a492e" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 42755 } health_report { overall_health: HEALTHY } } }
I20260806 06:54:37.924094 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.921694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.3.247.131:42755)
0806 06:54:37.921755 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:37.921762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:37.921772 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:37.921860 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:37.921861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:37.921881 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:37.921974 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:37.922165 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.922645 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.923420 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:37.923622 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:37.923690 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:37.923801 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:37.923886 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:37.923959 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:37.924001 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:54:37.924011 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:54:37.926409 22920 raft_consensus.cc:1261] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5 [term 1 FOLLOWER]: Refusing update from remote peer 10fff915f57643a6b5b52e8e4c0a492e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:37.926445 22789 raft_consensus.cc:1261] T 74058af1acb04fb88c5c48889d2cc0c0 P 3d726e07fede4598a6dab7210f97b0ff [term 1 FOLLOWER]: Refusing update from remote peer 10fff915f57643a6b5b52e8e4c0a492e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:37.926760 23121 consensus_queue.cc:1035] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d726e07fede4598a6dab7210f97b0ff" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:37.926899 23117 consensus_queue.cc:1035] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8acb30225144d2fb555a3015b94b8d5" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 40699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:37.929693 23124 mvcc.cc:205] Tried to move back new op lower bound from 7315453042383536128 to 7315453042311778304. Current Snapshot: MvccSnapshot[applied={T|T < 7315453042383536128 or (T in {7315453042383536128})}]
I20260806 06:54:37.930058 23126 mvcc.cc:205] Tried to move back new op lower bound from 7315453042383536128 to 7315453042311778304. Current Snapshot: MvccSnapshot[applied={T|T < 7315453042383536128}]
I20260806 06:54:37.931726 22647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:37.931824 22647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:37.932601 22647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b16bb0a46b845948a551ad3811946d6: Sending DeleteTablet for 3 replicas of tablet 74058af1acb04fb88c5c48889d2cc0c0
I20260806 06:54:37.932952 22769 tablet_service.cc:1514] Processing DeleteTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='slave'} at 127.0.0.1:58478
I20260806 06:54:37.932987 22900 tablet_service.cc:1514] Processing DeleteTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='slave'} at 127.0.0.1:46580
I20260806 06:54:37.933231 23031 tablet_service.cc:1514] Processing DeleteTablet for tablet 74058af1acb04fb88c5c48889d2cc0c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:37 UTC) from {username='slave'} at 127.0.0.1:57528
I20260806 06:54:37.933280 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22707
I20260806 06:54:37.934099 23136 tablet_replica.cc:330] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: stopping tablet replica
I20260806 06:54:37.934219 23136 raft_consensus.cc:2229] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:54:37.934350 23136 raft_consensus.cc:2258] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:37.934782 23136 ts_tablet_manager.cc:1905] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:37.936457 23136 ts_tablet_manager.cc:1918] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:37.936542 23136 log.cc:1196] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/74058af1acb04fb88c5c48889d2cc0c0
I20260806 06:54:37.936798 23136 ts_tablet_manager.cc:1939] T 74058af1acb04fb88c5c48889d2cc0c0 P 10fff915f57643a6b5b52e8e4c0a492e: Deleting consensus metadata
I20260806 06:54:37.937194 22634 catalog_manager.cc:4841] TS 10fff915f57643a6b5b52e8e4c0a492e (127.3.247.131:42755): tablet 74058af1acb04fb88c5c48889d2cc0c0 (table test-table [id=3099d27c7aab4ccb814902c3a66530f5]) successfully deleted
W20260806 06:54:37.941892 22633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:43085 (error 108)
I20260806 06:54:37.942081 23134 tablet_replica.cc:330] T 74058af1acb04fb88c5c48889d2cc0c0 P c8acb30225144d2fb555a3015b94b8d5: stopping tablet replica
I20260806 06:54:37.942112 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22838
I20260806 06:54:37.942809 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.942318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:43085 (local address 127.0.0.1:58494)
0806 06:54:37.942721 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.942761 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385}
W20260806 06:54:37.943169 22633 catalog_manager.cc:4584] TS 3d726e07fede4598a6dab7210f97b0ff (127.3.247.129:43085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74058af1acb04fb88c5c48889d2cc0c0: Network error: Client connection negotiation failed: client connection to 127.3.247.129:43085: connect: Connection refused (error 111)
I20260806 06:54:37.947206 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22969
I20260806 06:54:37.947314 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:37.947161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:40699 (local address 127.0.0.1:46588)
0806 06:54:37.947238 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:37.947272 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.130:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:54:37.952761 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 22616
2026-08-06T06:54:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (676 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:38 dist-test-slave-0kls krb5kdc[23140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:38 dist-test-slave-0kls krb5kdc[23140](info): set up 2 sockets
Aug 06 06:54:38 dist-test-slave-0kls krb5kdc[23140](info): commencing operation
krb5kdc: starting...
W20260806 06:54:39.013659 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:54:39.134644 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34023
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:34023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:39.217429 23153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:39.217653 23153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:39.217676 23153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:39.218941 23153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:39.218989 23153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:39.219003 23153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:39.219017 23153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:39.219029 23153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:39.220571 23153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:34023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:39.220847 23153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:39.221088 23153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:54:39.226008 23153 server_base.cc:1034] running on GCE node
W20260806 06:54:39.225982 23158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.225982 23161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.225976 23159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:39.226559 23153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:39.226855 23153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:39.228032 23153 hybrid_clock.cc:648] HybridClock initialized: now 1785999279228016 us; error 47 us; skew 500 ppm
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:39.230577 23153 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:39.231074 23153 webserver.cc:460] Webserver started at http://127.3.247.190:40669/ using document root <none> and password file <none>
I20260806 06:54:39.231321 23153 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:39.231395 23153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:39.231544 23153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:39.232597 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "b317b6dd64e2499b8af6bd0ed476639b"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "da2d1223e1285ac9946417d798fcf9be"
server_key_iv: "0595addc00019a6a5ab0d97fc676325f"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.232961 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "b317b6dd64e2499b8af6bd0ed476639b"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "da2d1223e1285ac9946417d798fcf9be"
server_key_iv: "0595addc00019a6a5ab0d97fc676325f"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.234452 23153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:39.235280 23168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.235488 23153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:39.235597 23153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "b317b6dd64e2499b8af6bd0ed476639b"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "da2d1223e1285ac9946417d798fcf9be"
server_key_iv: "0595addc00019a6a5ab0d97fc676325f"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.235675 23153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:39.255074 23153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:39.257010 23153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:39.257185 23153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:39.261574 23153 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:34023
I20260806 06:54:39.261634 23220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:34023 every 8 connection(s)
I20260806 06:54:39.262036 23153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:39.262917 23221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:39.265194 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: Bootstrap starting.
I20260806 06:54:39.265862 23221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:39.266168 23221 log.cc:824] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:39.266978 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: No bootstrap required, opened a new log
I20260806 06:54:39.268787 23221 raft_consensus.cc:348] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } }
I20260806 06:54:39.268940 23221 raft_consensus.cc:374] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:39.268966 23221 raft_consensus.cc:729] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b317b6dd64e2499b8af6bd0ed476639b, State: Initialized, Role: FOLLOWER
I20260806 06:54:39.269207 23221 consensus_queue.cc:260] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } }
I20260806 06:54:39.269297 23221 raft_consensus.cc:388] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:39.269322 23221 raft_consensus.cc:482] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:39.269402 23221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:39.269860 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23153
I20260806 06:54:39.270002 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:39.270247 4062 external_mini_cluster.cc:1428] Setting key f0073809cb0270e3be4e3dfdb2d6d394
I20260806 06:54:39.270313 23221 raft_consensus.cc:504] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } }
I20260806 06:54:39.270419 23221 leader_election.cc:302] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b317b6dd64e2499b8af6bd0ed476639b; no voters:
I20260806 06:54:39.270605 23221 leader_election.cc:288] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:39.270642 23224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:39.270819 23224 raft_consensus.cc:686] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [term 1 LEADER]: Becoming Leader. State: Replica: b317b6dd64e2499b8af6bd0ed476639b, State: Running, Role: LEADER
I20260806 06:54:39.270917 23221 sys_catalog.cc:564] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:39.270923 23224 consensus_queue.cc:237] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } }
I20260806 06:54:39.271332 23225 sys_catalog.cc:455] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b317b6dd64e2499b8af6bd0ed476639b. Latest consensus state: current_term: 1 leader_uuid: "b317b6dd64e2499b8af6bd0ed476639b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } } }
I20260806 06:54:39.271415 23225 sys_catalog.cc:458] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:39.271471 23227 sys_catalog.cc:455] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b317b6dd64e2499b8af6bd0ed476639b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b317b6dd64e2499b8af6bd0ed476639b" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34023 } } }
I20260806 06:54:39.271541 23227 sys_catalog.cc:458] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b [sys.catalog]: This master's current role is: LEADER
W20260806 06:54:39.272915 23242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:54:39.272998 23242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:54:39.273102 23233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:39.273250 23233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:39.274791 23233 catalog_manager.cc:1347] Generated new cluster ID: 80b85e5422e4478bb5f1cf48abaa141b
I20260806 06:54:39.274855 23233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:39.281839 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.270868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57734 (local address 127.3.247.190:34023)
0806 06:54:39.271627 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.271638 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.271682 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:54:39.271826 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.271832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.271875 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.272675 (+ 800us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:39.272907 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.274486 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.275384 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.275806 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.275927 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.278340 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:39.278344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:39.278345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:39.278357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:39.280169 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.280548 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.280551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.280552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.280663 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.280875 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.280876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.280876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.281140 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:39.281312 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.281423 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.281575 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":72,"threads_started":1}
I20260806 06:54:39.284682 23233 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:39.285458 23233 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:39.289351 23233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: Generated new TSK 0
I20260806 06:54:39.289552 23233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.247.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
Entry for principal kudu/127.3.247.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab.
I20260806 06:54:39.325057 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:39.408278 23247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:39.408491 23247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:39.408517 23247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:39.409797 23247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:39.409837 23247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:39.409865 23247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:39.411368 23247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.129.keytab
--principal=kudu/127.3.247.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:39.411616 23247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:39.411872 23247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:54:39.417102 23247 server_base.cc:1034] running on GCE node
W20260806 06:54:39.417068 23252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.417047 23255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.417075 23253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:39.417502 23247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:39.417819 23247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:39.418982 23247 hybrid_clock.cc:648] HybridClock initialized: now 1785999279418966 us; error 33 us; skew 500 ppm
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:39.421624 23247 init.cc:377] Logged in from keytab as kudu/127.3.247.129@KRBTEST.COM (short username kudu)
I20260806 06:54:39.422106 23247 webserver.cc:460] Webserver started at http://127.3.247.129:34315/ using document root <none> and password file <none>
I20260806 06:54:39.422358 23247 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:39.422403 23247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:39.422544 23247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:39.423604 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "350ef71ac7ae4bb3a2cf1d356e27d608"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "3a80f3430f30b5c93c6b6b8a9eae7047"
server_key_iv: "cc1b16fafc79a1b33cec5be33ed24a92"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.423964 23247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "350ef71ac7ae4bb3a2cf1d356e27d608"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "3a80f3430f30b5c93c6b6b8a9eae7047"
server_key_iv: "cc1b16fafc79a1b33cec5be33ed24a92"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.425362 23247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:39.426352 23262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.426534 23247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:54:39.426644 23247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "350ef71ac7ae4bb3a2cf1d356e27d608"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "3a80f3430f30b5c93c6b6b8a9eae7047"
server_key_iv: "cc1b16fafc79a1b33cec5be33ed24a92"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.426745 23247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:39.462260 23247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:39.464941 23247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:39.465128 23247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:39.465408 23247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:39.465764 23247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:39.465848 23247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.465896 23247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:39.465924 23247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.472707 23247 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:33143
I20260806 06:54:39.472764 23375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:33143 every 8 connection(s)
I20260806 06:54:39.473109 23247 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.129@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:39.480589 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23247
I20260806 06:54:39.480767 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:39.481020 4062 external_mini_cluster.cc:1428] Setting key 10aad969251a9fe3164141a0b4845a6d
I20260806 06:54:39.483563 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.473833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:42285 (local address 127.3.247.190:34023)
0806 06:54:39.473878 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.473880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.474180 (+ 300us) server_negotiation.cc:407] Connection header received
0806 06:54:39.474758 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.474759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.474777 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.474838 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:39.475893 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.476456 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.477559 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.477825 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.477928 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.480251 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:39.480253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:39.480253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:39.480260 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:39.482103 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.482506 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.482507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.482507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.482574 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.482822 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.482823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.482823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.482974 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:39.483047 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.483411 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.483495 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:39.483819 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.473841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34023 (local address 127.3.247.129:42285)
0806 06:54:39.474040 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.474081 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.474632 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.474939 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.474943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.475562 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:39.475779 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.475785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.476560 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.476561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.477442 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.477445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.477940 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.477941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.478084 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.478594 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:39.478603 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:39.480087 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:39.482185 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.482191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.482192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.482429 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.482641 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.482642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.482642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.482751 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.483038 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:39.483042 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:39.483212 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.483373 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.483552 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260806 06:54:39.484362 23376 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34023
I20260806 06:54:39.484474 23376 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:39.484683 23376 heartbeater.cc:511] Master 127.3.247.190:34023 requested a full tablet report, sending...
I20260806 06:54:39.485224 23185 ts_manager.cc:194] Registered new tserver with Master: 350ef71ac7ae4bb3a2cf1d356e27d608 (127.3.247.129:33143)
I20260806 06:54:39.485909 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.129@KRBTEST.COM'} at 127.3.247.129:42285
WARNING: no policy specified for kudu/127.3.247.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
Entry for principal kudu/127.3.247.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab.
I20260806 06:54:39.508406 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:39.589634 23381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:39.589963 23381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:39.590008 23381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:39.591192 23381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:39.591270 23381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:39.591318 23381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:39.592808 23381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.130.keytab
--principal=kudu/127.3.247.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:39.593135 23381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:39.593374 23381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:39.598381 23386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.598377 23387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.598378 23389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:39.598847 23381 server_base.cc:1034] running on GCE node
I20260806 06:54:39.599059 23381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:39.599294 23381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:39.600502 23381 hybrid_clock.cc:648] HybridClock initialized: now 1785999279600496 us; error 37 us; skew 500 ppm
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:39.603070 23381 init.cc:377] Logged in from keytab as kudu/127.3.247.130@KRBTEST.COM (short username kudu)
I20260806 06:54:39.603564 23381 webserver.cc:460] Webserver started at http://127.3.247.130:46385/ using document root <none> and password file <none>
I20260806 06:54:39.603811 23381 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:39.603857 23381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:39.604008 23381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:39.605602 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "3e667a782d1e472aab41b479c6854d76"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "d84c061b5fef7b72d36b10e705868e65"
server_key_iv: "94b7e355bb2c986443e83be904ae7e9a"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.606022 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "3e667a782d1e472aab41b479c6854d76"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "d84c061b5fef7b72d36b10e705868e65"
server_key_iv: "94b7e355bb2c986443e83be904ae7e9a"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.607368 23381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:39.608137 23396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.608314 23381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:39.608407 23381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "3e667a782d1e472aab41b479c6854d76"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "d84c061b5fef7b72d36b10e705868e65"
server_key_iv: "94b7e355bb2c986443e83be904ae7e9a"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.608482 23381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:39.619673 23381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:39.621515 23381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:39.621690 23381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:39.621951 23381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:39.622270 23381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:39.622303 23381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.622373 23381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:39.622404 23381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.628294 23381 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:44653
I20260806 06:54:39.628345 23509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:44653 every 8 connection(s)
I20260806 06:54:39.628656 23381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:54:39.633334 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23381
I20260806 06:54:39.633433 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:39.633651 4062 external_mini_cluster.cc:1428] Setting key f2662c3175c55158f9413acd2faca44f
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.130@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:39.638864 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.629388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:43057 (local address 127.3.247.190:34023)
0806 06:54:39.629437 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.629440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.629622 (+ 182us) server_negotiation.cc:407] Connection header received
0806 06:54:39.630268 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.630269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.630291 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.630341 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:39.631396 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.631918 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.632919 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.633156 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.633228 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.635842 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:39.635843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:39.635844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:39.635851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:39.637289 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.637724 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.637725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.637725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.637863 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.638139 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.638140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.638140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.638271 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:39.638347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.638679 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.638764 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:54:39.639072 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.629344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34023 (local address 127.3.247.130:43057)
0806 06:54:39.629489 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.629537 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.630158 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.630423 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.630427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.631102 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:39.631294 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.631299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.632014 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.632016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.632826 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.632828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.633247 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.633248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.633369 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.634067 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:39.634077 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:39.635733 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:39.637393 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.637400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.637403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.637637 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.637943 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.637944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.637945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.638068 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.638349 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:39.638357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:39.638541 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.638657 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.638853 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260806 06:54:39.639580 23510 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34023
I20260806 06:54:39.639688 23510 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:39.639889 23510 heartbeater.cc:511] Master 127.3.247.190:34023 requested a full tablet report, sending...
I20260806 06:54:39.640275 23185 ts_manager.cc:194] Registered new tserver with Master: 3e667a782d1e472aab41b479c6854d76 (127.3.247.130:44653)
I20260806 06:54:39.640651 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.130@KRBTEST.COM'} at 127.3.247.130:43057
WARNING: no policy specified for kudu/127.3.247.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
Entry for principal kudu/127.3.247.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab.
I20260806 06:54:39.660786 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:39.743901 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:39.744110 23515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:39.744134 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:39.745299 23515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:39.745344 23515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:39.745370 23515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:39.746767 23515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.131.keytab
--principal=kudu/127.3.247.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:39.746948 23515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:39.747116 23515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:39.752348 23523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.752388 23520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:39.752398 23521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:39.752542 23515 server_base.cc:1034] running on GCE node
I20260806 06:54:39.752820 23515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:39.753080 23515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:39.754257 23515 hybrid_clock.cc:648] HybridClock initialized: now 1785999279754242 us; error 40 us; skew 500 ppm
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:39.756807 23515 init.cc:377] Logged in from keytab as kudu/127.3.247.131@KRBTEST.COM (short username kudu)
I20260806 06:54:39.757228 23515 webserver.cc:460] Webserver started at http://127.3.247.131:36663/ using document root <none> and password file <none>
I20260806 06:54:39.757412 23515 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:39.757453 23515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:39.757544 23515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:39.758692 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "7cda388bcdda40559394013907654e92"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "a6c5ba4c1e16f880ac06e05fab003f23"
server_key_iv: "9723dc08dbc02b32688ce522c5556b98"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.759011 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "7cda388bcdda40559394013907654e92"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "a6c5ba4c1e16f880ac06e05fab003f23"
server_key_iv: "9723dc08dbc02b32688ce522c5556b98"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.760300 23515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:54:39.761061 23530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.761229 23515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:39.761315 23515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "7cda388bcdda40559394013907654e92"
format_stamp: "Formatted at 2026-08-06 06:54:39 on dist-test-slave-0kls"
server_key: "a6c5ba4c1e16f880ac06e05fab003f23"
server_key_iv: "9723dc08dbc02b32688ce522c5556b98"
server_key_version: "encryptionkey@0"
I20260806 06:54:39.761369 23515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:39.784770 23515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:39.787323 23515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:39.787452 23515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:39.787650 23515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:39.788004 23515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:39.788043 23515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.788065 23515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:39.788112 23515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:39.794068 23515 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:37671
I20260806 06:54:39.794169 23643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:37671 every 8 connection(s)
I20260806 06:54:39.794512 23515 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:39.795957 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23515
I20260806 06:54:39.796046 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:39.796242 4062 external_mini_cluster.cc:1428] Setting key 8cef9066343cd2aa862cca75812a1509
Aug 06 06:54:39 dist-test-slave-0kls krb5kdc[23140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999279, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.131@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:39.804208 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.795370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:37419 (local address 127.3.247.190:34023)
0806 06:54:39.795445 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.795447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.795669 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:54:39.796353 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.796354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.796368 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.796419 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:39.797392 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.798002 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.799016 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.799228 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.799324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.801530 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:39.801531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:39.801531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:39.801538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:39.802818 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.803183 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.803184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.803184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.803233 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.803641 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.803641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.803642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.803758 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:39.803839 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.804080 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.804142 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:54:39.804543 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.795348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:34023 (local address 127.3.247.131:37419)
0806 06:54:39.795542 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.795581 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.796206 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.796495 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.796499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.797084 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:39.797291 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.797295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.798097 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.798098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.798920 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.798922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.799327 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.799328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.799463 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.800015 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:54:39.800024 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:39.801439 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:39.802893 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.802899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.802901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.803119 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.803452 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:54:39.803453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:54:39.803453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:54:39.803573 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:54:39.803828 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:39.803833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:39.803958 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.804056 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.804225 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260806 06:54:39.805116 23644 heartbeater.cc:347] Connected to a master server at 127.3.247.190:34023
I20260806 06:54:39.805212 23644 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:39.805403 23644 heartbeater.cc:511] Master 127.3.247.190:34023 requested a full tablet report, sending...
I20260806 06:54:39.805765 23185 ts_manager.cc:194] Registered new tserver with Master: 7cda388bcdda40559394013907654e92 (127.3.247.131:37671)
I20260806 06:54:39.806182 23185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.247.131@KRBTEST.COM'} at 127.3.247.131:37419
I20260806 06:54:39.807928 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:39.815361 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.809202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.3.247.190:34023)
0806 06:54:39.809270 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.809273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.809429 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:54:39.809465 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.809466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.809474 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.809522 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:39.810172 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.810678 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.811506 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.811682 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.811732 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.812984 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:39.812985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:39.812986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:39.812992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:39.814314 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.814618 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.814619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.814619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.814668 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:39.814896 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:39.814897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:39.814897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:39.815032 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:39.815104 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.815232 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.815296 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:54:39.816921 23185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:39.818611 23185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:39.825259 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.821100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:33143 (local address 127.0.0.1:33522)
0806 06:54:39.821224 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.821234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.821280 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.821688 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.821691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.821693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:39.821908 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.821909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.822930 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.822932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.824313 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.824315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.824956 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.824957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.825033 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825046 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.825114 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.825164 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260806 06:54:39.825354 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.820889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44653 (local address 127.0.0.1:38596)
0806 06:54:39.821018 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.821061 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.821136 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.821709 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.821709 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.821711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:39.821835 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.821838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.823184 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.823184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.824295 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.824298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.825106 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.825107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.825160 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825162 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.825213 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.825247 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":31,"thread_start_us":62,"threads_started":1}
I20260806 06:54:39.825467 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.821311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37671 (local address 127.0.0.1:51470)
0806 06:54:39.821595 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.821604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.821652 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.821957 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.821958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.821960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:39.822002 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.822003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.823243 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.823243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.824175 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.824178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.825178 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.825179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.825241 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825246 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.825294 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.825334 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":64,"threads_started":1}
I20260806 06:54:39.825537 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.821423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.3.247.131:37671)
0806 06:54:39.821597 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.821602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.821709 (+ 107us) server_negotiation.cc:407] Connection header received
0806 06:54:39.821775 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.821777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.821818 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.821886 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:39.822074 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.823153 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.824266 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.825005 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.825081 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825140 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.825400 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.825456 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260806 06:54:39.825649 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.821216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.3.247.129:33143)
0806 06:54:39.821420 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.821425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.821435 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:39.821480 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.821481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.821493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.821591 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:39.821982 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.822811 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.824407 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.824868 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.824956 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825016 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.825501 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.825560 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260806 06:54:39.826282 23578 tablet_service.cc:1467] Processing CreateTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 (DEFAULT_TABLE table=test-table [id=d681d7bacf5d472d92e8497c72c7cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:39.826398 23310 tablet_service.cc:1467] Processing CreateTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 (DEFAULT_TABLE table=test-table [id=d681d7bacf5d472d92e8497c72c7cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:39.826660 23578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7952e973748d4cc4815b4ad83b4cc4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:39.826756 23310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7952e973748d4cc4815b4ad83b4cc4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:39.829262 23662 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Bootstrap starting.
I20260806 06:54:39.829340 23661 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Bootstrap starting.
I20260806 06:54:39.829458 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.821013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.3.247.130:44653)
0806 06:54:39.821471 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.821476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.821487 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:54:39.821527 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.821529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.821544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.821776 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:39.821895 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.823083 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.824382 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.825042 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.825104 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.825151 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.825224 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.825275 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":99,"threads_started":1}
I20260806 06:54:39.830058 23662 tablet_bootstrap.cc:654] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:39.830088 23661 tablet_bootstrap.cc:654] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:39.830124 23444 tablet_service.cc:1467] Processing CreateTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 (DEFAULT_TABLE table=test-table [id=d681d7bacf5d472d92e8497c72c7cb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:39.830525 23444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7952e973748d4cc4815b4ad83b4cc4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:39.830674 23662 log.cc:824] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:39.830732 23661 log.cc:824] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:39.831401 23661 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: No bootstrap required, opened a new log
I20260806 06:54:39.831452 23662 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: No bootstrap required, opened a new log
I20260806 06:54:39.831489 23661 ts_tablet_manager.cc:1397] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:54:39.831506 23662 ts_tablet_manager.cc:1397] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:54:39.833056 23661 raft_consensus.cc:348] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.833220 23661 raft_consensus.cc:374] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:39.833262 23661 raft_consensus.cc:729] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cda388bcdda40559394013907654e92, State: Initialized, Role: FOLLOWER
I20260806 06:54:39.833416 23661 consensus_queue.cc:260] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.833684 23661 ts_tablet_manager.cc:1428] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:54:39.833554 23662 raft_consensus.cc:348] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.833742 23662 raft_consensus.cc:374] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:39.833834 23662 raft_consensus.cc:729] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350ef71ac7ae4bb3a2cf1d356e27d608, State: Initialized, Role: FOLLOWER
I20260806 06:54:39.833846 23644 heartbeater.cc:503] Master 127.3.247.190:34023 was elected leader, sending a full tablet report...
I20260806 06:54:39.833971 23662 consensus_queue.cc:260] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.834390 23662 ts_tablet_manager.cc:1428] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:54:39.834390 23376 heartbeater.cc:503] Master 127.3.247.190:34023 was elected leader, sending a full tablet report...
I20260806 06:54:39.834491 23666 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Bootstrap starting.
I20260806 06:54:39.835191 23666 tablet_bootstrap.cc:654] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:39.835770 23666 log.cc:824] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:39.836457 23666 tablet_bootstrap.cc:492] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: No bootstrap required, opened a new log
I20260806 06:54:39.836511 23666 ts_tablet_manager.cc:1397] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:39.838150 23666 raft_consensus.cc:348] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.838299 23666 raft_consensus.cc:374] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:39.838321 23666 raft_consensus.cc:729] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e667a782d1e472aab41b479c6854d76, State: Initialized, Role: FOLLOWER
I20260806 06:54:39.838402 23666 consensus_queue.cc:260] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.838663 23666 ts_tablet_manager.cc:1428] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:54:39.838711 23510 heartbeater.cc:503] Master 127.3.247.190:34023 was elected leader, sending a full tablet report...
W20260806 06:54:39.879276 23511 tablet.cc:2369] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:39.957525 23667 raft_consensus.cc:482] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:39.957660 23667 raft_consensus.cc:504] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.958151 23667 leader_election.cc:288] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e667a782d1e472aab41b479c6854d76 (127.3.247.130:44653), 7cda388bcdda40559394013907654e92 (127.3.247.131:37671)
I20260806 06:54:39.961344 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.958475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:40841 (local address 127.3.247.130:44653)
0806 06:54:39.958539 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.958541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.958567 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:54:39.958629 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.958631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.958640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.958734 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:39.958950 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.959532 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.960574 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.960943 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.961001 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.961036 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.961226 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.961272 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:54:39.961344 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.958727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:46499 (local address 127.3.247.131:37671)
0806 06:54:39.958791 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:39.958793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:39.958804 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:54:39.958872 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:39.958874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:39.958885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:39.958949 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:39.959109 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.959592 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.960541 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.960945 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.961046 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.961055 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:39.961220 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:54:39.961261 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:54:39.961373 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.958572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:37671 (local address 127.3.247.129:46499)
0806 06:54:39.958738 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.958750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.958813 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.958958 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.958958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.958960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:39.959031 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.959033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.959692 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.959693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.960435 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.960436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.961056 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.961057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.961135 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.961138 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.961210 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.961252 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260806 06:54:39.961375 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:39.958417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:44653 (local address 127.3.247.129:40841)
0806 06:54:39.958488 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:39.958501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:39.958555 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:39.958756 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:39.958757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:39.958759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:39.958841 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.958844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.959629 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.959639 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.960441 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:39.960443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:39.961034 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:39.961035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:39.961125 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:39.961129 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:39.961208 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:54:39.961251 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:54:39.961757 23598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7952e973748d4cc4815b4ad83b4cc4f0" candidate_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cda388bcdda40559394013907654e92" is_pre_election: true
I20260806 06:54:39.961954 23598 raft_consensus.cc:2454] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 350ef71ac7ae4bb3a2cf1d356e27d608 in term 0.
I20260806 06:54:39.962054 23464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7952e973748d4cc4815b4ad83b4cc4f0" candidate_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e667a782d1e472aab41b479c6854d76" is_pre_election: true
I20260806 06:54:39.962271 23464 raft_consensus.cc:2454] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 350ef71ac7ae4bb3a2cf1d356e27d608 in term 0.
I20260806 06:54:39.962222 23266 leader_election.cc:302] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350ef71ac7ae4bb3a2cf1d356e27d608, 7cda388bcdda40559394013907654e92; no voters:
I20260806 06:54:39.962417 23667 raft_consensus.cc:2781] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:39.962476 23667 raft_consensus.cc:482] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:39.962512 23667 raft_consensus.cc:3037] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:39.963235 23667 raft_consensus.cc:504] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.963379 23667 leader_election.cc:288] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [CANDIDATE]: Term 1 election: Requested vote from peers 3e667a782d1e472aab41b479c6854d76 (127.3.247.130:44653), 7cda388bcdda40559394013907654e92 (127.3.247.131:37671)
I20260806 06:54:39.963630 23598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7952e973748d4cc4815b4ad83b4cc4f0" candidate_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cda388bcdda40559394013907654e92"
I20260806 06:54:39.963611 23464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7952e973748d4cc4815b4ad83b4cc4f0" candidate_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e667a782d1e472aab41b479c6854d76"
I20260806 06:54:39.963707 23598 raft_consensus.cc:3037] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:39.963707 23464 raft_consensus.cc:3037] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:39.964629 23598 raft_consensus.cc:2454] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 350ef71ac7ae4bb3a2cf1d356e27d608 in term 1.
I20260806 06:54:39.964653 23464 raft_consensus.cc:2454] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 350ef71ac7ae4bb3a2cf1d356e27d608 in term 1.
I20260806 06:54:39.964802 23266 leader_election.cc:302] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350ef71ac7ae4bb3a2cf1d356e27d608, 7cda388bcdda40559394013907654e92; no voters:
I20260806 06:54:39.964881 23667 raft_consensus.cc:2781] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:39.965049 23667 raft_consensus.cc:686] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [term 1 LEADER]: Becoming Leader. State: Replica: 350ef71ac7ae4bb3a2cf1d356e27d608, State: Running, Role: LEADER
I20260806 06:54:39.965153 23667 consensus_queue.cc:237] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } }
I20260806 06:54:39.965958 23185 catalog_manager.cc:5495] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 reported cstate change: term changed from 0 to 1, leader changed from <none> to 350ef71ac7ae4bb3a2cf1d356e27d608 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "350ef71ac7ae4bb3a2cf1d356e27d608" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 33143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 } health_report { overall_health: UNKNOWN } } }
W20260806 06:54:39.973935 23377 tablet.cc:2369] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:40.007525 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.003792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33534 (local address 127.3.247.129:33143)
0806 06:54:40.003840 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.003842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.003853 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:54:40.003922 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.003923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.003947 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.004054 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:40.004254 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.005081 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.006420 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.006785 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.006862 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.007054 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:40.007232 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:40.007339 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.007385 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.007409 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:54:40.009769 23598 raft_consensus.cc:1261] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 1 FOLLOWER]: Refusing update from remote peer 350ef71ac7ae4bb3a2cf1d356e27d608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:40.010090 23464 raft_consensus.cc:1261] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 1 FOLLOWER]: Refusing update from remote peer 350ef71ac7ae4bb3a2cf1d356e27d608: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:40.010202 23671 consensus_queue.cc:1035] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cda388bcdda40559394013907654e92" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:40.010478 23667 consensus_queue.cc:1035] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e667a782d1e472aab41b479c6854d76" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 44653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:40.013345 23674 mvcc.cc:205] Tried to move back new op lower bound from 7315453050916851712 to 7315453050737946624. Current Snapshot: MvccSnapshot[applied={T|T < 7315453050916851712}]
I20260806 06:54:40.015838 23184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:40.015957 23184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:40.016736 23184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b317b6dd64e2499b8af6bd0ed476639b: Sending DeleteTablet for 3 replicas of tablet 7952e973748d4cc4815b4ad83b4cc4f0
I20260806 06:54:40.017086 23444 tablet_service.cc:1514] Processing DeleteTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:38596
I20260806 06:54:40.017136 23310 tablet_service.cc:1514] Processing DeleteTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:33522
I20260806 06:54:40.017211 23578 tablet_service.cc:1514] Processing DeleteTablet for tablet 7952e973748d4cc4815b4ad83b4cc4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='kudu', principal='kudu/127.3.247.190@KRBTEST.COM'} at 127.0.0.1:51470
I20260806 06:54:40.017418 23686 tablet_replica.cc:330] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: stopping tablet replica
I20260806 06:54:40.017524 23686 raft_consensus.cc:2229] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:40.017554 23684 tablet_replica.cc:330] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: stopping tablet replica
I20260806 06:54:40.017639 23686 raft_consensus.cc:2258] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:40.017741 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23247
I20260806 06:54:40.017769 23684 raft_consensus.cc:2229] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:40.018807 23686 ts_tablet_manager.cc:1905] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:40.018822 23684 raft_consensus.cc:2258] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:40.017431 23685 tablet_replica.cc:330] T 7952e973748d4cc4815b4ad83b4cc4f0 P 350ef71ac7ae4bb3a2cf1d356e27d608: stopping tablet replica
I20260806 06:54:40.019529 23684 ts_tablet_manager.cc:1905] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:40.020577 23686 ts_tablet_manager.cc:1918] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:40.020663 23686 log.cc:1196] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-2/wal/wals/7952e973748d4cc4815b4ad83b4cc4f0
I20260806 06:54:40.020967 23686 ts_tablet_manager.cc:1939] T 7952e973748d4cc4815b4ad83b4cc4f0 P 7cda388bcdda40559394013907654e92: Deleting consensus metadata
I20260806 06:54:40.021396 23172 catalog_manager.cc:4841] TS 7cda388bcdda40559394013907654e92 (127.3.247.131:37671): tablet 7952e973748d4cc4815b4ad83b4cc4f0 (table test-table [id=d681d7bacf5d472d92e8497c72c7cb6c]) successfully deleted
I20260806 06:54:40.021504 23684 ts_tablet_manager.cc:1918] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:40.021580 23684 log.cc:1196] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/7952e973748d4cc4815b4ad83b4cc4f0
I20260806 06:54:40.021883 23684 ts_tablet_manager.cc:1939] T 7952e973748d4cc4815b4ad83b4cc4f0 P 3e667a782d1e472aab41b479c6854d76: Deleting consensus metadata
I20260806 06:54:40.022312 23172 catalog_manager.cc:4841] TS 3e667a782d1e472aab41b479c6854d76 (127.3.247.130:44653): tablet 7952e973748d4cc4815b4ad83b4cc4f0 (table test-table [id=d681d7bacf5d472d92e8497c72c7cb6c]) successfully deleted
W20260806 06:54:40.024000 23172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:33143 (error 108)
I20260806 06:54:40.024510 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23381
I20260806 06:54:40.024590 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.024400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:33143 (local address 127.0.0.1:33544)
0806 06:54:40.024495 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.024534 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 06:54:40.025094 23172 catalog_manager.cc:4584] TS 350ef71ac7ae4bb3a2cf1d356e27d608 (127.3.247.129:33143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7952e973748d4cc4815b4ad83b4cc4f0: Network error: Client connection negotiation failed: client connection to 127.3.247.129:33143: connect: Connection refused (error 111)
I20260806 06:54:40.029326 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23515
I20260806 06:54:40.034297 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23153
2026-08-06T06:54:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2081 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T06:54:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:40Z Disabled control of system clock
I20260806 06:54:40.062733 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:37001
--webserver_interface=127.3.247.190
--webserver_port=0
--builtin_ntp_servers=127.3.247.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:37001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:40.143800 23691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:40.144026 23691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:40.144060 23691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:40.145159 23691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:40.145192 23691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:40.145205 23691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:40.145217 23691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:40.145228 23691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:40.146526 23691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:37001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:37001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:40.146720 23691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:40.146944 23691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:40.151661 23696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.151684 23699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.151700 23697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.152022 23691 server_base.cc:1034] running on GCE node
I20260806 06:54:40.152251 23691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:40.152490 23691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:40.153621 23691 hybrid_clock.cc:648] HybridClock initialized: now 1785999280153611 us; error 32 us; skew 500 ppm
I20260806 06:54:40.154853 23691 webserver.cc:460] Webserver started at http://127.3.247.190:46471/ using document root <none> and password file <none>
I20260806 06:54:40.155103 23691 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:40.155148 23691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:40.155292 23691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:40.156385 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "e9e4cbeff03e45629c33d3519e470852"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "0c328abc45e801bb6d9c9d631677fe14"
server_key_iv: "cd8987625cff4d525a201d2f88c77386"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.156765 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "e9e4cbeff03e45629c33d3519e470852"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "0c328abc45e801bb6d9c9d631677fe14"
server_key_iv: "cd8987625cff4d525a201d2f88c77386"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.158154 23691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:40.158910 23705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.159114 23691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:40.159185 23691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "e9e4cbeff03e45629c33d3519e470852"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "0c328abc45e801bb6d9c9d631677fe14"
server_key_iv: "cd8987625cff4d525a201d2f88c77386"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.159282 23691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:40.174145 23691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:40.174494 23691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:40.174650 23691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:40.178690 23691 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:37001
I20260806 06:54:40.178736 23757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:37001 every 8 connection(s)
I20260806 06:54:40.179112 23691 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:40.180064 23758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:40.182386 23758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: Bootstrap starting.
I20260806 06:54:40.182984 23758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:40.183290 23758 log.cc:824] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:40.183965 23758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: No bootstrap required, opened a new log
I20260806 06:54:40.185526 23758 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } }
I20260806 06:54:40.185664 23758 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:40.185714 23758 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e4cbeff03e45629c33d3519e470852, State: Initialized, Role: FOLLOWER
I20260806 06:54:40.185894 23758 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } }
I20260806 06:54:40.185977 23758 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:40.185999 23758 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:40.186034 23758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:40.186506 23758 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } }
I20260806 06:54:40.186587 23758 leader_election.cc:302] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9e4cbeff03e45629c33d3519e470852; no voters:
I20260806 06:54:40.186736 23758 leader_election.cc:288] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:40.186774 23761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:40.186914 23761 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [term 1 LEADER]: Becoming Leader. State: Replica: e9e4cbeff03e45629c33d3519e470852, State: Running, Role: LEADER
I20260806 06:54:40.187008 23758 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:40.187094 23761 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } }
I20260806 06:54:40.187253 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23691
I20260806 06:54:40.187368 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:40.187590 4062 external_mini_cluster.cc:1428] Setting key 2618a0966fc22b9147b6b7493c5dd43e
I20260806 06:54:40.187620 23762 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9e4cbeff03e45629c33d3519e470852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } } }
I20260806 06:54:40.187704 23762 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:40.188118 23763 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9e4cbeff03e45629c33d3519e470852. Latest consensus state: current_term: 1 leader_uuid: "e9e4cbeff03e45629c33d3519e470852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e4cbeff03e45629c33d3519e470852" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 37001 } } }
I20260806 06:54:40.188237 23763 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:40.188340 23770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:40.189144 23770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:40.190866 23770 catalog_manager.cc:1347] Generated new cluster ID: d8452a43772d42498cd6265b89493a69
I20260806 06:54:40.190927 23770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:40.194089 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.188319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.3.247.190:37001)
0806 06:54:40.188542 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.188552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.188591 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:54:40.188811 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.188817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.188882 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.189042 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:40.189315 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.190644 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.191846 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.192484 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.192613 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.193346 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:40.193351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:40.193351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:40.193555 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:40.193595 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:40.193610 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:40.193680 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.193762 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.193857 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260806 06:54:40.215029 23770 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:40.215601 23770 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:40.220050 23770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: Generated new TSK 0
I20260806 06:54:40.220273 23770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:40.228431 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.129:0
--local_ip_for_outbound_sockets=127.3.247.129
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--builtin_ntp_servers=127.3.247.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:40.317481 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:40.317734 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:40.317770 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:40.318979 23782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:40.319051 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:40.319094 23782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.129
I20260806 06:54:40.320473 23782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.247.129
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.129
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:40.320746 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:40.321048 23782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:40.326373 23787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.326489 23782 server_base.cc:1034] running on GCE node
W20260806 06:54:40.326455 23788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.326463 23790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.326833 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:40.327099 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:40.328269 23782 hybrid_clock.cc:648] HybridClock initialized: now 1785999280328262 us; error 36 us; skew 500 ppm
I20260806 06:54:40.329597 23782 webserver.cc:460] Webserver started at http://127.3.247.129:41439/ using document root <none> and password file <none>
I20260806 06:54:40.329907 23782 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:40.329980 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:40.330143 23782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:40.331252 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data/instance:
uuid: "ce91edeccba64e29a30d445acced1e10"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "8f8be28609db05251b5dd72a8146fdd7"
server_key_iv: "c4d50a9b938d183b38b35f01f17f8b43"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.331619 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance:
uuid: "ce91edeccba64e29a30d445acced1e10"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "8f8be28609db05251b5dd72a8146fdd7"
server_key_iv: "c4d50a9b938d183b38b35f01f17f8b43"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.332924 23782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:40.333649 23796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.333850 23782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:40.333953 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal
uuid: "ce91edeccba64e29a30d445acced1e10"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "8f8be28609db05251b5dd72a8146fdd7"
server_key_iv: "c4d50a9b938d183b38b35f01f17f8b43"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.334043 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:40.345026 23782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:40.345315 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:40.345453 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:40.345667 23782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:40.346021 23782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:40.346073 23782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.346095 23782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:40.346143 23782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.351538 23782 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.129:37879
I20260806 06:54:40.351629 23909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.129:37879 every 8 connection(s)
I20260806 06:54:40.351949 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/data/info.pb
I20260806 06:54:40.354265 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23782
I20260806 06:54:40.354382 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-0/wal/instance
I20260806 06:54:40.354612 4062 external_mini_cluster.cc:1428] Setting key a5a1c8ac23f12f0f3177fd00ab6cd7fd
I20260806 06:54:40.355726 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.130:0
--local_ip_for_outbound_sockets=127.3.247.130
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--builtin_ntp_servers=127.3.247.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:54:40.358409 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.352725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:49549 (local address 127.3.247.190:37001)
0806 06:54:40.352785 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.352787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.353039 (+ 252us) server_negotiation.cc:407] Connection header received
0806 06:54:40.353587 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.353589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.353604 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.353662 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:40.354623 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.355405 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.356909 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.357111 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.357197 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.358095 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:40.358097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:40.358097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:40.358102 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:40.358117 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:40.358128 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:40.358189 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.358299 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.358331 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:54:40.358615 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.352725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37001 (local address 127.3.247.129:49549)
0806 06:54:40.352921 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.352954 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.353478 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.353735 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.353738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.353746 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:40.354519 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.354527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.355511 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.355512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.356815 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.356819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.357206 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.357207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.357340 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.357942 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:40.357943 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:40.357973 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:40.357995 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:40.358190 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:40.358196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:40.358200 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.358281 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.358384 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260806 06:54:40.359077 23910 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37001
I20260806 06:54:40.359182 23910 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:40.359381 23910 heartbeater.cc:511] Master 127.3.247.190:37001 requested a full tablet report, sending...
I20260806 06:54:40.359889 23722 ts_manager.cc:194] Registered new tserver with Master: ce91edeccba64e29a30d445acced1e10 (127.3.247.129:37879)
I20260806 06:54:40.360493 23722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.129:49549
W20260806 06:54:40.439133 23913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:40.439365 23913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:40.439391 23913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:40.440701 23913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:40.440748 23913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:40.440776 23913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.130
I20260806 06:54:40.442281 23913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.247.130
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.130
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:40.442548 23913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:40.442878 23913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:40.448091 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.448112 23921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.448091 23918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.448168 23913 server_base.cc:1034] running on GCE node
I20260806 06:54:40.448485 23913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:40.448734 23913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:40.449868 23913 hybrid_clock.cc:648] HybridClock initialized: now 1785999280449859 us; error 30 us; skew 500 ppm
I20260806 06:54:40.451207 23913 webserver.cc:460] Webserver started at http://127.3.247.130:45747/ using document root <none> and password file <none>
I20260806 06:54:40.451436 23913 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:40.451479 23913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:40.451573 23913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:40.452663 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data/instance:
uuid: "ca86401c942845bb80fa5096ec0091dd"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "9035bbade637d69e56b2c6695c5da6a5"
server_key_iv: "cf9b89eb653817ff5089952c3a78b013"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.452973 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance:
uuid: "ca86401c942845bb80fa5096ec0091dd"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "9035bbade637d69e56b2c6695c5da6a5"
server_key_iv: "cf9b89eb653817ff5089952c3a78b013"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.454334 23913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:40.455085 23927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.455266 23913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:40.455364 23913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal
uuid: "ca86401c942845bb80fa5096ec0091dd"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "9035bbade637d69e56b2c6695c5da6a5"
server_key_iv: "cf9b89eb653817ff5089952c3a78b013"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.455448 23913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:40.467892 23913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:40.468219 23913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:40.468372 23913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:40.468601 23913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:40.468936 23913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:40.468987 23913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.469023 23913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:40.469049 23913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.474723 23913 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.130:41385
I20260806 06:54:40.474848 24040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.130:41385 every 8 connection(s)
I20260806 06:54:40.475167 23913 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/data/info.pb
I20260806 06:54:40.480159 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.475945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.130:44209 (local address 127.3.247.190:37001)
0806 06:54:40.475990 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.475992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.476211 (+ 219us) server_negotiation.cc:407] Connection header received
0806 06:54:40.476681 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.476682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.476693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.476744 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:40.477429 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.477992 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.478948 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.479140 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.479193 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.479947 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:40.479948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:40.479948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:40.479952 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:40.479965 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:40.479968 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:40.480014 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.480077 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.480100 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:54:40.480340 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.475940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37001 (local address 127.3.247.130:44209)
0806 06:54:40.476093 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.476127 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.476599 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.476805 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.476808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.476816 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:40.477358 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.477362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.478089 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.478090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.478874 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.478876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.479217 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.479217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.479329 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.479824 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:40.479824 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:40.479851 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:40.479872 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:40.480016 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:40.480019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:40.480021 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.480066 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.480141 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260806 06:54:40.480955 24041 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37001
I20260806 06:54:40.481035 24041 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:40.481233 24041 heartbeater.cc:511] Master 127.3.247.190:37001 requested a full tablet report, sending...
I20260806 06:54:40.481420 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 23913
I20260806 06:54:40.481523 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal/instance
I20260806 06:54:40.481640 23722 ts_manager.cc:194] Registered new tserver with Master: ca86401c942845bb80fa5096ec0091dd (127.3.247.130:41385)
I20260806 06:54:40.481740 4062 external_mini_cluster.cc:1428] Setting key ba1f9187cc1dfcb47c98ec4376778c8f
I20260806 06:54:40.482235 23722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.130:44209
I20260806 06:54:40.482743 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.247.131:0
--local_ip_for_outbound_sockets=127.3.247.131
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--builtin_ntp_servers=127.3.247.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:54:40.564688 24044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:40.564935 24044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:40.564985 24044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:40.566239 24044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:40.566298 24044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:40.566350 24044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.247.131
I20260806 06:54:40.567682 24044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.247.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.247.131
--webserver_port=0
--tserver_master_addrs=127.3.247.190:37001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.247.131
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:40.567942 24044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:40.568257 24044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:40.573416 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:40.573432 24050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.573496 24044 server_base.cc:1034] running on GCE node
W20260806 06:54:40.573432 24049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:40.573868 24044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:40.574095 24044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:40.575232 24044 hybrid_clock.cc:648] HybridClock initialized: now 1785999280575216 us; error 31 us; skew 500 ppm
I20260806 06:54:40.576468 24044 webserver.cc:460] Webserver started at http://127.3.247.131:36565/ using document root <none> and password file <none>
I20260806 06:54:40.576711 24044 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:40.576759 24044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:40.576928 24044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:40.578086 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data/instance:
uuid: "17c050423ff544a5b1ea84291756cc6c"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "60b6f8649b97a6197e0c8f6fcb95a809"
server_key_iv: "88517afdc4b69a993f4c1abd6cd2be69"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.578445 24044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance:
uuid: "17c050423ff544a5b1ea84291756cc6c"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "60b6f8649b97a6197e0c8f6fcb95a809"
server_key_iv: "88517afdc4b69a993f4c1abd6cd2be69"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.579810 24044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:40.580597 24058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.580760 24044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:40.580850 24044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal
uuid: "17c050423ff544a5b1ea84291756cc6c"
format_stamp: "Formatted at 2026-08-06 06:54:40 on dist-test-slave-0kls"
server_key: "60b6f8649b97a6197e0c8f6fcb95a809"
server_key_iv: "88517afdc4b69a993f4c1abd6cd2be69"
server_key_version: "encryptionkey@0"
I20260806 06:54:40.580909 24044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:40.595072 24044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:40.595345 24044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:40.595448 24044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:40.595633 24044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:54:40.595969 24044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:54:40.596004 24044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.596058 24044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:54:40.596086 24044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:40.601652 24044 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.131:40383
I20260806 06:54:40.601732 24171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.131:40383 every 8 connection(s)
I20260806 06:54:40.602080 24044 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/data/info.pb
I20260806 06:54:40.607391 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.602863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.131:39887 (local address 127.3.247.190:37001)
0806 06:54:40.602910 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.602912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.603177 (+ 265us) server_negotiation.cc:407] Connection header received
0806 06:54:40.603722 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.603723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.603736 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.603774 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:40.604468 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.604970 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.606030 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.606226 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.606331 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.607101 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:40.607103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:40.607103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:40.607108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:40.607122 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:40.607126 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:40.607178 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.607275 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.607310 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:54:40.607395 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24044
I20260806 06:54:40.607522 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-2/wal/instance
I20260806 06:54:40.607604 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.602845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:37001 (local address 127.3.247.131:39887)
0806 06:54:40.603049 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.603085 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.603609 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.603869 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.603872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.603880 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:54:40.604366 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.604370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.605077 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.605078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.605930 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.605932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.606339 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.606340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.606477 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.606968 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:54:40.606968 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:54:40.606996 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:54:40.607017 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:54:40.607186 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:54:40.607191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:54:40.607193 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.607277 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.607378 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260806 06:54:40.607735 4062 external_mini_cluster.cc:1428] Setting key 4a9cd24eb1bd8c335426a545e1bf8223
I20260806 06:54:40.608299 24172 heartbeater.cc:347] Connected to a master server at 127.3.247.190:37001
I20260806 06:54:40.608489 24172 heartbeater.cc:464] Registering TS with master...
I20260806 06:54:40.608683 24172 heartbeater.cc:511] Master 127.3.247.190:37001 requested a full tablet report, sending...
I20260806 06:54:40.609063 23722 ts_manager.cc:194] Registered new tserver with Master: 17c050423ff544a5b1ea84291756cc6c (127.3.247.131:40383)
I20260806 06:54:40.609443 23722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.247.131:39887
I20260806 06:54:40.619822 4062 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:54:40.624351 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.621220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.3.247.190:37001)
0806 06:54:40.621306 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.621309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.621402 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:54:40.621495 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.621496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.621510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.621568 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:40.621742 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.622299 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.623072 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.623278 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.623355 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.624066 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:40.624067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:40.624067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:40.624072 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:40.624087 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:40.624091 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:40.624161 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.624223 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.624249 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:54:40.625988 23722 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:54:40.627538 23722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:54:40.633873 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.629722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37879 (local address 127.0.0.1:54970)
0806 06:54:40.629957 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.629996 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.630061 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.630324 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.630326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.630329 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:40.630397 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.630399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.631521 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.631522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.632600 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.632602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.633512 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.633594 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.633608 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.633691 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.633749 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":124,"threads_started":1}
I20260806 06:54:40.633876 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.629962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.3.247.129:37879)
0806 06:54:40.630144 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.630148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.630157 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:54:40.630195 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.630203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.630215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.630318 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:40.630553 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.631398 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.632679 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.633409 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633497 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.633585 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.633716 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.633765 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260806 06:54:40.634368 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.629722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:40383 (local address 127.0.0.1:38388)
0806 06:54:40.630249 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.630258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.630336 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.630560 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.630561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.630564 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:40.630609 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.630610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.631965 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.631966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.633065 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.633067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.634120 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.634121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.634184 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.634188 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.634245 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.634295 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":42,"spinlock_wait_cycles":768,"thread_start_us":139,"threads_started":1}
I20260806 06:54:40.634375 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.629930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.3.247.131:40383)
0806 06:54:40.630117 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.630122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.630338 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:54:40.630406 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.630408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.630423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.630494 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:40.630685 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.631875 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633152 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.633922 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633993 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.634082 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.634259 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.634295 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":82,"threads_started":1}
I20260806 06:54:40.634699 23844 tablet_service.cc:1467] Processing CreateTablet for tablet f100199eb67b4802ba5fcfc070682828 (DEFAULT_TABLE table=test-table [id=5f4a53a89d2641ad8feadf92fd19493f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:40.634982 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.629736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:41385 (local address 127.0.0.1:41892)
0806 06:54:40.630414 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.630424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.630470 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.630733 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.630734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.630736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:40.630770 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.630771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.632083 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.632084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.632991 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.632993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.633854 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.634272 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.634278 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.634339 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.634384 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":2,"thread_start_us":122,"threads_started":1}
I20260806 06:54:40.634877 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.630022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.3.247.130:41385)
0806 06:54:40.630191 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.630194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.630533 (+ 339us) server_negotiation.cc:407] Connection header received
0806 06:54:40.630578 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.630580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.630597 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.630665 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:40.630858 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.632000 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633072 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.633660 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.633722 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.633822 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.634755 (+ 933us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.634800 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260806 06:54:40.635109 23844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f100199eb67b4802ba5fcfc070682828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:40.635236 24106 tablet_service.cc:1467] Processing CreateTablet for tablet f100199eb67b4802ba5fcfc070682828 (DEFAULT_TABLE table=test-table [id=5f4a53a89d2641ad8feadf92fd19493f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:40.635581 24106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f100199eb67b4802ba5fcfc070682828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:40.635694 23975 tablet_service.cc:1467] Processing CreateTablet for tablet f100199eb67b4802ba5fcfc070682828 (DEFAULT_TABLE table=test-table [id=5f4a53a89d2641ad8feadf92fd19493f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:54:40.636014 23975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f100199eb67b4802ba5fcfc070682828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:40.638587 24186 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: Bootstrap starting.
I20260806 06:54:40.639178 24186 tablet_bootstrap.cc:654] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:40.639330 24188 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Bootstrap starting.
I20260806 06:54:40.639494 24187 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: Bootstrap starting.
I20260806 06:54:40.639536 24186 log.cc:824] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:40.639976 24188 tablet_bootstrap.cc:654] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:40.640069 24187 tablet_bootstrap.cc:654] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:40.640278 24186 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: No bootstrap required, opened a new log
I20260806 06:54:40.640313 24188 log.cc:824] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:40.640359 24186 ts_tablet_manager.cc:1397] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:40.640378 24187 log.cc:824] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:40.641057 24188 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: No bootstrap required, opened a new log
I20260806 06:54:40.641063 24187 tablet_bootstrap.cc:492] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: No bootstrap required, opened a new log
I20260806 06:54:40.641120 24187 ts_tablet_manager.cc:1397] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:40.641120 24188 ts_tablet_manager.cc:1397] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:54:40.642596 24186 raft_consensus.cc:348] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.642753 24186 raft_consensus.cc:374] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:40.642800 24186 raft_consensus.cc:729] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c050423ff544a5b1ea84291756cc6c, State: Initialized, Role: FOLLOWER
I20260806 06:54:40.642933 24186 consensus_queue.cc:260] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.643205 24186 ts_tablet_manager.cc:1428] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:54:40.643340 24172 heartbeater.cc:503] Master 127.3.247.190:37001 was elected leader, sending a full tablet report...
I20260806 06:54:40.643653 24188 raft_consensus.cc:348] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.643653 24187 raft_consensus.cc:348] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.643813 24187 raft_consensus.cc:374] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:40.643813 24188 raft_consensus.cc:374] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:40.643849 24188 raft_consensus.cc:729] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca86401c942845bb80fa5096ec0091dd, State: Initialized, Role: FOLLOWER
I20260806 06:54:40.643849 24187 raft_consensus.cc:729] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce91edeccba64e29a30d445acced1e10, State: Initialized, Role: FOLLOWER
I20260806 06:54:40.643978 24187 consensus_queue.cc:260] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.643978 24188 consensus_queue.cc:260] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.644243 24187 ts_tablet_manager.cc:1428] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:54:40.644243 24188 ts_tablet_manager.cc:1428] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:54:40.644356 23910 heartbeater.cc:503] Master 127.3.247.190:37001 was elected leader, sending a full tablet report...
I20260806 06:54:40.644559 24041 heartbeater.cc:503] Master 127.3.247.190:37001 was elected leader, sending a full tablet report...
W20260806 06:54:40.725708 24042 tablet.cc:2369] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:54:40.747531 24193 raft_consensus.cc:482] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:54:40.747663 24193 raft_consensus.cc:504] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.748112 24193 leader_election.cc:288] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17c050423ff544a5b1ea84291756cc6c (127.3.247.131:40383), ca86401c942845bb80fa5096ec0091dd (127.3.247.130:41385)
I20260806 06:54:40.751303 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.748476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:46409 (local address 127.3.247.130:41385)
0806 06:54:40.748580 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.748582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.748592 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:54:40.748624 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.748625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.748635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.748698 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:40.749007 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.749550 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.750544 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.750928 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.750984 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.750991 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.751159 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.751207 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 06:54:40.751305 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.748333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.130:41385 (local address 127.3.247.129:46409)
0806 06:54:40.748421 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.748435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.748494 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.748820 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.748821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.748823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:40.748901 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.748904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.749634 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.749635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.750446 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.750449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.751006 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.751007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.751078 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.751082 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.751137 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.751172 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 06:54:40.751354 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.748479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.247.129:52041 (local address 127.3.247.131:40383)
0806 06:54:40.748527 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.748529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.748806 (+ 277us) server_negotiation.cc:407] Connection header received
0806 06:54:40.748839 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.748840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.748849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.748915 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:54:40.749136 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.749613 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.750606 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.750982 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.751055 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.751063 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.751237 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.751283 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:54:40.751329 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.748334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:40383 (local address 127.3.247.129:52041)
0806 06:54:40.748562 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.748574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:54:40.748657 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:54:40.748978 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:54:40.748979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:54:40.748982 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:54:40.749057 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.749059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.749695 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.749695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.750509 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:54:40.750512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.751071 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.751072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:54:40.751152 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.751155 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:54:40.751213 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:54:40.751248 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":24,"thread_start_us":101,"threads_started":1}
I20260806 06:54:40.751771 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f100199eb67b4802ba5fcfc070682828" candidate_uuid: "ce91edeccba64e29a30d445acced1e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca86401c942845bb80fa5096ec0091dd" is_pre_election: true
I20260806 06:54:40.751964 23995 raft_consensus.cc:2454] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce91edeccba64e29a30d445acced1e10 in term 0.
I20260806 06:54:40.751935 24126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f100199eb67b4802ba5fcfc070682828" candidate_uuid: "ce91edeccba64e29a30d445acced1e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c050423ff544a5b1ea84291756cc6c" is_pre_election: true
I20260806 06:54:40.752058 24126 raft_consensus.cc:2454] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce91edeccba64e29a30d445acced1e10 in term 0.
I20260806 06:54:40.752301 23797 leader_election.cc:302] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c050423ff544a5b1ea84291756cc6c, ce91edeccba64e29a30d445acced1e10; no voters:
I20260806 06:54:40.752446 24193 raft_consensus.cc:2781] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:54:40.752504 24193 raft_consensus.cc:482] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:54:40.752527 24193 raft_consensus.cc:3037] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:40.753252 24193 raft_consensus.cc:504] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.753407 24193 leader_election.cc:288] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [CANDIDATE]: Term 1 election: Requested vote from peers 17c050423ff544a5b1ea84291756cc6c (127.3.247.131:40383), ca86401c942845bb80fa5096ec0091dd (127.3.247.130:41385)
I20260806 06:54:40.753583 24126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f100199eb67b4802ba5fcfc070682828" candidate_uuid: "ce91edeccba64e29a30d445acced1e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c050423ff544a5b1ea84291756cc6c"
I20260806 06:54:40.753585 23995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f100199eb67b4802ba5fcfc070682828" candidate_uuid: "ce91edeccba64e29a30d445acced1e10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca86401c942845bb80fa5096ec0091dd"
I20260806 06:54:40.753669 24126 raft_consensus.cc:3037] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:40.753669 23995 raft_consensus.cc:3037] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:40.754631 24126 raft_consensus.cc:2454] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce91edeccba64e29a30d445acced1e10 in term 1.
I20260806 06:54:40.754631 23995 raft_consensus.cc:2454] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce91edeccba64e29a30d445acced1e10 in term 1.
I20260806 06:54:40.754814 23800 leader_election.cc:302] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca86401c942845bb80fa5096ec0091dd, ce91edeccba64e29a30d445acced1e10; no voters:
I20260806 06:54:40.754936 24193 raft_consensus.cc:2781] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:40.755082 24193 raft_consensus.cc:686] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 1 LEADER]: Becoming Leader. State: Replica: ce91edeccba64e29a30d445acced1e10, State: Running, Role: LEADER
I20260806 06:54:40.755156 24193 consensus_queue.cc:237] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } }
I20260806 06:54:40.756029 23722 catalog_manager.cc:5495] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce91edeccba64e29a30d445acced1e10 (127.3.247.129). New cstate: current_term: 1 leader_uuid: "ce91edeccba64e29a30d445acced1e10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce91edeccba64e29a30d445acced1e10" member_type: VOTER last_known_addr { host: "127.3.247.129" port: 37879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 } health_report { overall_health: UNKNOWN } } }
I20260806 06:54:40.778285 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.775855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.3.247.129:37879)
0806 06:54:40.775917 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:40.775919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:40.775928 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:54:40.775972 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:40.775973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:40.775991 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:40.776069 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:54:40.776179 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.776665 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.777438 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:40.777632 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:40.777738 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:40.777918 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:54:40.778029 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:54:40.778121 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:40.778147 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:54:40.778165 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:54:40.780077 23995 raft_consensus.cc:1261] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 1 FOLLOWER]: Refusing update from remote peer ce91edeccba64e29a30d445acced1e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:40.780099 24126 raft_consensus.cc:1261] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 1 FOLLOWER]: Refusing update from remote peer ce91edeccba64e29a30d445acced1e10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:54:40.780432 24193 consensus_queue.cc:1035] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca86401c942845bb80fa5096ec0091dd" member_type: VOTER last_known_addr { host: "127.3.247.130" port: 41385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:40.780557 24196 consensus_queue.cc:1035] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c050423ff544a5b1ea84291756cc6c" member_type: VOTER last_known_addr { host: "127.3.247.131" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:54:40.782910 24199 mvcc.cc:205] Tried to move back new op lower bound from 7315453054072655872 to 7315453053973856256. Current Snapshot: MvccSnapshot[applied={T|T < 7315453054072655872}]
I20260806 06:54:40.783522 24200 mvcc.cc:205] Tried to move back new op lower bound from 7315453054072655872 to 7315453053973856256. Current Snapshot: MvccSnapshot[applied={T|T < 7315453054072655872}]
I20260806 06:54:40.784341 24201 mvcc.cc:205] Tried to move back new op lower bound from 7315453054072655872 to 7315453053973856256. Current Snapshot: MvccSnapshot[applied={T|T < 7315453054072655872 or (T in {7315453054072655872})}]
I20260806 06:54:40.785465 23722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:40.785544 23722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:54:40.786299 23722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9e4cbeff03e45629c33d3519e470852: Sending DeleteTablet for 3 replicas of tablet f100199eb67b4802ba5fcfc070682828
I20260806 06:54:40.786643 23975 tablet_service.cc:1514] Processing DeleteTablet for tablet f100199eb67b4802ba5fcfc070682828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='slave'} at 127.0.0.1:41892
I20260806 06:54:40.786671 24106 tablet_service.cc:1514] Processing DeleteTablet for tablet f100199eb67b4802ba5fcfc070682828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='slave'} at 127.0.0.1:38388
I20260806 06:54:40.786903 24208 tablet_replica.cc:330] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: stopping tablet replica
I20260806 06:54:40.786921 23844 tablet_service.cc:1514] Processing DeleteTablet for tablet f100199eb67b4802ba5fcfc070682828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:54:40 UTC) from {username='slave'} at 127.0.0.1:54970
I20260806 06:54:40.787060 24208 raft_consensus.cc:2229] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:40.787168 24208 raft_consensus.cc:2258] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:40.787312 24209 tablet_replica.cc:330] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c: stopping tablet replica
I20260806 06:54:40.787325 24210 tablet_replica.cc:330] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10: stopping tablet replica
I20260806 06:54:40.787403 24209 raft_consensus.cc:2229] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:54:40.787418 24210 raft_consensus.cc:2229] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:54:40.787493 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23782
I20260806 06:54:40.787537 24209 raft_consensus.cc:2258] T f100199eb67b4802ba5fcfc070682828 P 17c050423ff544a5b1ea84291756cc6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:40.787549 24210 raft_consensus.cc:2258] T f100199eb67b4802ba5fcfc070682828 P ce91edeccba64e29a30d445acced1e10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:54:40.789233 24208 ts_tablet_manager.cc:1905] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:54:40.790788 24208 ts_tablet_manager.cc:1918] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:54:40.790880 24208 log.cc:1196] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Deleting WAL directory at /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785999040366271-4062-0/minicluster-data/ts-1/wal/wals/f100199eb67b4802ba5fcfc070682828
I20260806 06:54:40.791123 24208 ts_tablet_manager.cc:1939] T f100199eb67b4802ba5fcfc070682828 P ca86401c942845bb80fa5096ec0091dd: Deleting consensus metadata
I20260806 06:54:40.791532 23709 catalog_manager.cc:4841] TS ca86401c942845bb80fa5096ec0091dd (127.3.247.130:41385): tablet f100199eb67b4802ba5fcfc070682828 (table test-table [id=5f4a53a89d2641ad8feadf92fd19493f]) successfully deleted
W20260806 06:54:40.792611 23707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.247.129:37879 (error 108)
I20260806 06:54:40.792816 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23913
I20260806 06:54:40.793670 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.793515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.129:37879 (local address 127.0.0.1:54974)
0806 06:54:40.793580 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.793628 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:54:40.793845 23707 catalog_manager.cc:4584] TS ce91edeccba64e29a30d445acced1e10 (127.3.247.129:37879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f100199eb67b4802ba5fcfc070682828: Network error: Client connection negotiation failed: client connection to 127.3.247.129:37879: connect: Connection refused (error 111)
I20260806 06:54:40.798207 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24044
I20260806 06:54:40.802708 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 23691
I20260806 06:54:40.802809 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:40.802675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.131:40383 (local address 127.0.0.1:38404)
0806 06:54:40.802725 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:54:40.802765 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.131:40383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
2026-08-06T06:54:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (767 ms)
[----------] 6 tests from EncryptionPolicyTest (8640 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:40 dist-test-slave-0kls krb5kdc[24214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:40 dist-test-slave-0kls krb5kdc[24214](info): set up 2 sockets
Aug 06 06:54:40 dist-test-slave-0kls krb5kdc[24214](info): commencing operation
krb5kdc: starting...
W20260806 06:54:41.838063 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:41 dist-test-slave-0kls krb5kdc[24214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:41.956797 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:42.046422 24227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:42.046626 24227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:42.046648 24227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:42.047817 24227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:42.047858 24227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:42.047873 24227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:42.047884 24227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:42.047896 24227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:42.049412 24227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:42.049606 24227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:42.049777 24227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:42.054934 24235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:42.054910 24233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:42.054915 24232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:42.055009 24227 server_base.cc:1034] running on GCE node
I20260806 06:54:42.055402 24227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:42.055714 24227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:42.056885 24227 hybrid_clock.cc:648] HybridClock initialized: now 1785999282056880 us; error 42 us; skew 500 ppm
Aug 06 06:54:42 dist-test-slave-0kls krb5kdc[24214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999282, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:42.059441 24227 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:42.059890 24227 webserver.cc:460] Webserver started at http://127.0.0.1:46773/ using document root <none> and password file <none>
I20260806 06:54:42.060122 24227 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:42.060190 24227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:42.060351 24227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:42.061446 24227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "db167ecd118f4368b476dd0aeb9722bf"
format_stamp: "Formatted at 2026-08-06 06:54:42 on dist-test-slave-0kls"
server_key: "ecec5bd09f34abdf91a03df9529f8aeb"
server_key_iv: "ecba3817086280549ee5f4287f9724a9"
server_key_version: "encryptionkey@0"
I20260806 06:54:42.061877 24227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "db167ecd118f4368b476dd0aeb9722bf"
format_stamp: "Formatted at 2026-08-06 06:54:42 on dist-test-slave-0kls"
server_key: "ecec5bd09f34abdf91a03df9529f8aeb"
server_key_iv: "ecba3817086280549ee5f4287f9724a9"
server_key_version: "encryptionkey@0"
I20260806 06:54:42.063233 24227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:42.064010 24242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:42.064179 24227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:42.064303 24227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "db167ecd118f4368b476dd0aeb9722bf"
format_stamp: "Formatted at 2026-08-06 06:54:42 on dist-test-slave-0kls"
server_key: "ecec5bd09f34abdf91a03df9529f8aeb"
server_key_iv: "ecba3817086280549ee5f4287f9724a9"
server_key_version: "encryptionkey@0"
I20260806 06:54:42.064391 24227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:42.093287 24227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:42.095368 24227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:42.095549 24227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:42.099694 24227 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40357
I20260806 06:54:42.099735 24294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40357 every 8 connection(s)
I20260806 06:54:42.100116 24227 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:42.101121 24295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:42.102023 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24227
I20260806 06:54:42.102144 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:42.102352 4062 external_mini_cluster.cc:1428] Setting key c6c671fab51e81f5bb8a17d378b5a0c1
I20260806 06:54:42.104246 24295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf: Bootstrap starting.
I20260806 06:54:42.104857 24295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:42.105134 24295 log.cc:824] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:42.105870 24295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf: No bootstrap required, opened a new log
I20260806 06:54:42.107822 24295 raft_consensus.cc:348] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } }
I20260806 06:54:42.107983 24295 raft_consensus.cc:374] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:42.108034 24295 raft_consensus.cc:729] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db167ecd118f4368b476dd0aeb9722bf, State: Initialized, Role: FOLLOWER
I20260806 06:54:42.108186 24295 consensus_queue.cc:260] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } }
I20260806 06:54:42.108283 24295 raft_consensus.cc:388] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:42.108327 24295 raft_consensus.cc:482] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:42.108376 24295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:54:42 dist-test-slave-0kls krb5kdc[24214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:54:42.109104 24295 raft_consensus.cc:504] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } }
I20260806 06:54:42.109203 24295 leader_election.cc:302] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db167ecd118f4368b476dd0aeb9722bf; no voters:
I20260806 06:54:42.109413 24295 leader_election.cc:288] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:42.109481 24300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:42.109635 24300 raft_consensus.cc:686] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [term 1 LEADER]: Becoming Leader. State: Replica: db167ecd118f4368b476dd0aeb9722bf, State: Running, Role: LEADER
I20260806 06:54:42.109686 24295 sys_catalog.cc:564] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:42.109814 24300 consensus_queue.cc:237] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } }
I20260806 06:54:42.110160 24300 sys_catalog.cc:455] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader db167ecd118f4368b476dd0aeb9722bf. Latest consensus state: current_term: 1 leader_uuid: "db167ecd118f4368b476dd0aeb9722bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } } }
I20260806 06:54:42.110255 24300 sys_catalog.cc:458] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:42.110175 24301 sys_catalog.cc:455] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db167ecd118f4368b476dd0aeb9722bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db167ecd118f4368b476dd0aeb9722bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40357 } } }
I20260806 06:54:42.110423 24301 sys_catalog.cc:458] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:42.110562 24307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:42.111111 24307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:42.112843 24307 catalog_manager.cc:1347] Generated new cluster ID: 67c1523f475845d0973fa17ae681ef72
I20260806 06:54:42.112895 24307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:42.114369 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:42.102923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49899 (local address 127.0.0.1:40357)
0806 06:54:42.103129 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:42.103137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:42.103163 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:54:42.103584 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:42.103590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:42.103635 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:42.103781 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:42.104468 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:42.105460 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:42.106373 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:42.106790 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:42.106919 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:42.109876 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:42.109880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:42.109881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:42.109894 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:42.112632 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:42.113132 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:42.113135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:42.113135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:42.113236 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:42.113491 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:42.113492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:42.113492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:42.113741 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:42.113984 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:42.114075 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:54:42.114181 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260806 06:54:42.118854 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:42.128309 24298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:42.120280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46369 (local address 127.0.0.1:40357)
0806 06:54:42.120330 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:42.120332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:42.122027 (+ 1695us) server_negotiation.cc:407] Connection header received
0806 06:54:42.122103 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:42.122104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:42.122114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:42.122158 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:42.122692 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:42.123283 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:42.124014 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:42.124239 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:42.124333 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:42.125601 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:42.125603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:42.125603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:42.125611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:42.127083 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:42.127456 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:42.127458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:42.127458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:42.127521 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:42.127746 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:42.127748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:42.127748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:42.127864 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:42.127991 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:42.128109 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 06:54:42.128175 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:54:42.135696 24307 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:42.136427 24307 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:42.148213 24307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db167ecd118f4368b476dd0aeb9722bf: Generated new TSK 0
I20260806 06:54:42.148463 24307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:42.158454 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24227
W20260806 06:54:42.163218 24212 connection.cc:535] client connection to 127.0.0.1:40357 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40357): Connection reset by peer (error 104)
2026-08-06T06:54:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:42 dist-test-slave-0kls krb5kdc[24326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:42 dist-test-slave-0kls krb5kdc[24326](info): set up 2 sockets
Aug 06 06:54:42 dist-test-slave-0kls krb5kdc[24326](info): commencing operation
krb5kdc: starting...
W20260806 06:54:43.182237 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:43 dist-test-slave-0kls krb5kdc[24326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:43.298748 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:43.380280 24339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:43.380525 24339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:43.380568 24339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:43.381907 24339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:43.381983 24339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:43.382007 24339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:43.382028 24339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:43.382069 24339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:43.383548 24339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:43.383833 24339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:43.384063 24339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:43.388795 24344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:43.388824 24347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:43.388856 24345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:43.389066 24339 server_base.cc:1034] running on GCE node
I20260806 06:54:43.389340 24339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:43.389642 24339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:43.390843 24339 hybrid_clock.cc:648] HybridClock initialized: now 1785999283390832 us; error 51 us; skew 500 ppm
Aug 06 06:54:43 dist-test-slave-0kls krb5kdc[24326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:43.396607 24339 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:43.397116 24339 webserver.cc:460] Webserver started at http://127.0.0.1:33473/ using document root <none> and password file <none>
I20260806 06:54:43.397372 24339 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:43.397444 24339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:43.397593 24339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:43.398759 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "0c62b96429fc48b88a0388dcef82af03"
format_stamp: "Formatted at 2026-08-06 06:54:43 on dist-test-slave-0kls"
server_key: "5c21904c0f1256e36d4f789be4e14a2a"
server_key_iv: "8be7e59dfb28d389c96067ee18b9f8c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:43.399124 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "0c62b96429fc48b88a0388dcef82af03"
format_stamp: "Formatted at 2026-08-06 06:54:43 on dist-test-slave-0kls"
server_key: "5c21904c0f1256e36d4f789be4e14a2a"
server_key_iv: "8be7e59dfb28d389c96067ee18b9f8c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:43.400462 24339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:43.401196 24354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:43.401369 24339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:43.401458 24339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "0c62b96429fc48b88a0388dcef82af03"
format_stamp: "Formatted at 2026-08-06 06:54:43 on dist-test-slave-0kls"
server_key: "5c21904c0f1256e36d4f789be4e14a2a"
server_key_iv: "8be7e59dfb28d389c96067ee18b9f8c4"
server_key_version: "encryptionkey@0"
I20260806 06:54:43.401539 24339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:43.413574 24339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:43.416071 24339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:43.416251 24339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:43.420454 24339 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33015
I20260806 06:54:43.420490 24406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33015 every 8 connection(s)
I20260806 06:54:43.420843 24339 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:43.421947 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:43.423498 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24339
I20260806 06:54:43.423614 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:43.423840 4062 external_mini_cluster.cc:1428] Setting key 760bba6625387cc9476552b1cecb6000
I20260806 06:54:43.424635 24407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: Bootstrap starting.
I20260806 06:54:43.425230 24407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:43.425594 24407 log.cc:824] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:43.426357 24407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: No bootstrap required, opened a new log
I20260806 06:54:43.428664 24407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } }
I20260806 06:54:43.428864 24407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:43.428910 24407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62b96429fc48b88a0388dcef82af03, State: Initialized, Role: FOLLOWER
I20260806 06:54:43.429049 24407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } }
I20260806 06:54:43.429149 24407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:43.429193 24407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:43.429240 24407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:43.429710 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:43.424387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37559 (local address 127.0.0.1:33015)
0806 06:54:43.424568 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:43.424576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:43.424601 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:54:43.425058 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:43.425066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:43.425115 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:43.425292 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:43.425551 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:43.426689 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:43.427752 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:43.428131 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:43.428221 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:43.428998 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:43.429002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:43.429003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:43.429206 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:43.429247 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:43.429262 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:43.429342 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:43.429466 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:54:43.429496 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260806 06:54:43.429917 24407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } }
I20260806 06:54:43.430022 24407 leader_election.cc:302] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c62b96429fc48b88a0388dcef82af03; no voters:
I20260806 06:54:43.430181 24407 leader_election.cc:288] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:43.430203 24412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:43.430346 24412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62b96429fc48b88a0388dcef82af03, State: Running, Role: LEADER
I20260806 06:54:43.430431 24407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:43.430451 24412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } }
I20260806 06:54:43.430678 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:43.431584 24414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c62b96429fc48b88a0388dcef82af03. Latest consensus state: current_term: 1 leader_uuid: "0c62b96429fc48b88a0388dcef82af03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } } }
I20260806 06:54:43.431670 24414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [sys.catalog]: This master's current role is: LEADER
W20260806 06:54:43.432271 24426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:54:43.432466 24426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:54:43.432588 24429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:43.432564 24413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c62b96429fc48b88a0388dcef82af03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c62b96429fc48b88a0388dcef82af03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33015 } } }
I20260806 06:54:43.432726 24413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:43.432790 24429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:43.434365 24429 catalog_manager.cc:1347] Generated new cluster ID: e6f95e0be7794a6b8177da8753862e8d
I20260806 06:54:43.434418 24429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:43.436792 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:43.432401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34693 (local address 127.0.0.1:33015)
0806 06:54:43.432443 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:43.432445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:43.433011 (+ 566us) server_negotiation.cc:407] Connection header received
0806 06:54:43.433058 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:43.433059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:43.433069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:43.433135 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:43.433296 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:43.434053 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:43.435194 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:43.435623 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:43.435682 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:43.436530 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:43.436531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:43.436531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:43.436535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:43.436548 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:43.436553 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:43.436582 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:43.436697 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 06:54:43.436707 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:54:43.439432 24429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:43.440286 24429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:43.446074 24429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c62b96429fc48b88a0388dcef82af03: Generated new TSK 0
I20260806 06:54:43.446259 24429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:43.454275 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24339
W20260806 06:54:43.458698 24324 connection.cc:535] client connection to 127.0.0.1:33015 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33015): Connection reset by peer (error 104)
2026-08-06T06:54:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:43 dist-test-slave-0kls krb5kdc[24438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:43 dist-test-slave-0kls krb5kdc[24438](info): set up 2 sockets
Aug 06 06:54:43 dist-test-slave-0kls krb5kdc[24438](info): commencing operation
krb5kdc: starting...
W20260806 06:54:44.505990 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:44 dist-test-slave-0kls krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:44.629194 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:44.711834 24451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:44.712121 24451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:44.712181 24451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:44.713551 24451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:44.713634 24451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:44.713665 24451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:44.713693 24451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:44.713732 24451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:44.715495 24451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:44.715822 24451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:44.716063 24451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:44.721525 24457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:44.721519 24459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:44.721524 24456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:44.721556 24451 server_base.cc:1034] running on GCE node
I20260806 06:54:44.721978 24451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:44.722301 24451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:44.723449 24451 hybrid_clock.cc:648] HybridClock initialized: now 1785999284723440 us; error 32 us; skew 500 ppm
Aug 06 06:54:44 dist-test-slave-0kls krb5kdc[24438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:44.726017 24451 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:44.726476 24451 webserver.cc:460] Webserver started at http://127.0.0.1:45229/ using document root <none> and password file <none>
I20260806 06:54:44.726713 24451 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:44.726780 24451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:44.726938 24451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:44.727974 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51"
format_stamp: "Formatted at 2026-08-06 06:54:44 on dist-test-slave-0kls"
server_key: "b4a3bb8add13b096b441bb4cf4189ebe"
server_key_iv: "3462c90bc28fb68b19f744c735867c42"
server_key_version: "encryptionkey@0"
I20260806 06:54:44.728336 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51"
format_stamp: "Formatted at 2026-08-06 06:54:44 on dist-test-slave-0kls"
server_key: "b4a3bb8add13b096b441bb4cf4189ebe"
server_key_iv: "3462c90bc28fb68b19f744c735867c42"
server_key_version: "encryptionkey@0"
I20260806 06:54:44.729619 24451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:54:44.730481 24466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:44.730665 24451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:44.730763 24451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51"
format_stamp: "Formatted at 2026-08-06 06:54:44 on dist-test-slave-0kls"
server_key: "b4a3bb8add13b096b441bb4cf4189ebe"
server_key_iv: "3462c90bc28fb68b19f744c735867c42"
server_key_version: "encryptionkey@0"
I20260806 06:54:44.730857 24451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:44.734136 24451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:44.736390 24451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:44.736546 24451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:44.740448 24451 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35545
I20260806 06:54:44.740476 24518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35545 every 8 connection(s)
I20260806 06:54:44.740794 24451 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:44.741736 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:44.743888 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24451
I20260806 06:54:44.744017 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:44.744133 24519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51: Bootstrap starting.
I20260806 06:54:44.744231 4062 external_mini_cluster.cc:1428] Setting key 9e8991a0f7399abc9e6b9166de32b494
I20260806 06:54:44.744839 24519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:44.745136 24519 log.cc:824] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:44.745872 24519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51: No bootstrap required, opened a new log
I20260806 06:54:44.746742 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:44.744706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49473 (local address 127.0.0.1:35545)
0806 06:54:44.744919 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:44.744929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:44.744960 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:54:44.745446 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:44.745452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:44.745479 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:44.745625 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:44.746336 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:44.746340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:44.746340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:44.746344 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:44.746389 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:44.746400 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:44.746490 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:44.746534 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:54:44.746573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260806 06:54:44.747673 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:44.748136 24519 raft_consensus.cc:348] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260806 06:54:44.748266 24519 raft_consensus.cc:374] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:44.748298 24519 raft_consensus.cc:729] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d44d9ca8160649ea8f73f5fe1d3c6d51, State: Initialized, Role: FOLLOWER
I20260806 06:54:44.748435 24519 consensus_queue.cc:260] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260806 06:54:44.748517 24519 raft_consensus.cc:388] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:44.748538 24519 raft_consensus.cc:482] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:44.748601 24519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:44.749313 24519 raft_consensus.cc:504] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260806 06:54:44.749429 24519 leader_election.cc:302] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d44d9ca8160649ea8f73f5fe1d3c6d51; no voters:
I20260806 06:54:44.749588 24519 leader_election.cc:288] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:44.749621 24529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:44.749778 24529 raft_consensus.cc:686] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [term 1 LEADER]: Becoming Leader. State: Replica: d44d9ca8160649ea8f73f5fe1d3c6d51, State: Running, Role: LEADER
I20260806 06:54:44.749852 24519 sys_catalog.cc:564] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:44.749915 24529 consensus_queue.cc:237] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } }
I20260806 06:54:44.750240 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:44.749057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52623 (local address 127.0.0.1:35545)
0806 06:54:44.749091 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:44.749093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:44.749212 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:54:44.749281 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:44.749282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:44.749285 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:44.749328 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:44.750002 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:44.750003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:44.750004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:44.750009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:44.750022 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:44.750026 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:44.750084 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:44.750168 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:54:44.750181 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:54:44.750257 24529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d44d9ca8160649ea8f73f5fe1d3c6d51. Latest consensus state: current_term: 1 leader_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260806 06:54:44.750274 24530 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44d9ca8160649ea8f73f5fe1d3c6d51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35545 } } }
I20260806 06:54:44.750350 24529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:44.750392 24530 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:44.750617 24536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:44.751519 24536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:44.752948 24536 catalog_manager.cc:1347] Generated new cluster ID: 3a30e423efce4db69f7ec8af9628f989
I20260806 06:54:44.752996 24536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:44.772704 24536 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:44.772912 24536 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:44.784559 24536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d44d9ca8160649ea8f73f5fe1d3c6d51: Generated new TSK 0
I20260806 06:54:44.784796 24536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:44.789399 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24451
2026-08-06T06:54:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:44 dist-test-slave-0kls krb5kdc[24550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:44 dist-test-slave-0kls krb5kdc[24550](info): set up 2 sockets
Aug 06 06:54:44 dist-test-slave-0kls krb5kdc[24550](info): commencing operation
krb5kdc: starting...
W20260806 06:54:45.806038 4062 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:45 dist-test-slave-0kls krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:45.922703 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:46.002899 24563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:46.003170 24563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:46.003212 24563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:46.005318 24563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:46.005369 24563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:46.005396 24563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:46.005419 24563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:46.005448 24563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:46.007675 24563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:46.007943 24563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:46.008188 24563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:46.013729 24568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:46.013810 24569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:46.013736 24571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:46.014019 24563 server_base.cc:1034] running on GCE node
I20260806 06:54:46.014248 24563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:46.014569 24563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:46.015735 24563 hybrid_clock.cc:648] HybridClock initialized: now 1785999286015726 us; error 40 us; skew 500 ppm
Aug 06 06:54:46 dist-test-slave-0kls krb5kdc[24550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:46.018142 24563 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:46.018620 24563 webserver.cc:460] Webserver started at http://127.0.0.1:45853/ using document root <none> and password file <none>
I20260806 06:54:46.018859 24563 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:46.018931 24563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:46.019093 24563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:46.020167 24563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "3c7058f4896c46729b8f3b2f878b7435"
format_stamp: "Formatted at 2026-08-06 06:54:46 on dist-test-slave-0kls"
server_key: "8aa3a26ac28fb324cd62724ab17a4982"
server_key_iv: "01004dd47268ee7bc57796393e986139"
server_key_version: "encryptionkey@0"
I20260806 06:54:46.020541 24563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "3c7058f4896c46729b8f3b2f878b7435"
format_stamp: "Formatted at 2026-08-06 06:54:46 on dist-test-slave-0kls"
server_key: "8aa3a26ac28fb324cd62724ab17a4982"
server_key_iv: "01004dd47268ee7bc57796393e986139"
server_key_version: "encryptionkey@0"
I20260806 06:54:46.021929 24563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260806 06:54:46.022645 24578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:46.022825 24563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:46.022922 24563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "3c7058f4896c46729b8f3b2f878b7435"
format_stamp: "Formatted at 2026-08-06 06:54:46 on dist-test-slave-0kls"
server_key: "8aa3a26ac28fb324cd62724ab17a4982"
server_key_iv: "01004dd47268ee7bc57796393e986139"
server_key_version: "encryptionkey@0"
I20260806 06:54:46.023010 24563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:46.043432 24563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:46.045302 24563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:46.045477 24563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:46.049638 24563 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37275
I20260806 06:54:46.049671 24630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37275 every 8 connection(s)
I20260806 06:54:46.050072 24563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:46.051076 24631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:46.053274 24631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435: Bootstrap starting.
I20260806 06:54:46.053853 24631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:46.054129 24631 log.cc:824] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:46.054767 24631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435: No bootstrap required, opened a new log
I20260806 06:54:46.056253 24631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 06:54:46.056387 24631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:46.056409 24631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7058f4896c46729b8f3b2f878b7435, State: Initialized, Role: FOLLOWER
I20260806 06:54:46.056510 24631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 06:54:46.056562 24631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:46.056583 24631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:46.056610 24631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:46.057096 24631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 06:54:46.057178 24631 leader_election.cc:302] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c7058f4896c46729b8f3b2f878b7435; no voters:
I20260806 06:54:46.057327 24631 leader_election.cc:288] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:46.057374 24634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:46.057575 24634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [term 1 LEADER]: Becoming Leader. State: Replica: 3c7058f4896c46729b8f3b2f878b7435, State: Running, Role: LEADER
I20260806 06:54:46.057583 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24563
I20260806 06:54:46.057621 24631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:46.057680 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:46.057672 24634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 06:54:46.057936 4062 external_mini_cluster.cc:1428] Setting key a0898840e8a5990ee74858609b5063a8
I20260806 06:54:46.058025 24634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c7058f4896c46729b8f3b2f878b7435. Latest consensus state: current_term: 1 leader_uuid: "3c7058f4896c46729b8f3b2f878b7435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } } }
I20260806 06:54:46.058108 24634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:46.058262 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c7058f4896c46729b8f3b2f878b7435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7058f4896c46729b8f3b2f878b7435" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } } }
I20260806 06:54:46.058405 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:46.058329 24640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:46.059062 24640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:46.060994 24640 catalog_manager.cc:1347] Generated new cluster ID: 6c68aec85b8d47029f678ce46b212fb8
I20260806 06:54:46.061050 24640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:54:46 dist-test-slave-0kls krb5kdc[24550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:54:46.068591 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:46.058721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45565 (local address 127.0.0.1:37275)
0806 06:54:46.058876 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:46.058885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:46.058918 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:54:46.059038 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:46.059046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:46.059153 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:46.060097 (+ 944us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:46.060170 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:46.061466 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:46.062293 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:46.062709 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:46.062860 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:46.065207 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:46.065212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:46.065212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:46.065225 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:46.067008 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:46.067399 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:46.067402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:46.067403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:46.067507 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:46.067721 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:46.067722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:46.067722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:46.067971 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:46.068151 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:46.068221 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:54:46.068368 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260806 06:54:46.069588 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:46.075520 24640 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:46.076277 24640 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:46.077293 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:46.070957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47251 (local address 127.0.0.1:37275)
0806 06:54:46.070995 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:46.070997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:46.071119 (+ 122us) server_negotiation.cc:407] Connection header received
0806 06:54:46.071193 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:46.071194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:46.071205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:46.071258 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:46.071764 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:46.072263 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:46.072982 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:46.073195 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:46.073285 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:46.074477 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:46.074479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:46.074479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:46.074486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:46.076136 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:46.076527 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:46.076528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:46.076529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:46.076579 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:46.076751 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:46.076751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:46.076751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:46.076854 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:46.076933 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:46.077113 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:54:46.077194 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:46.079242 24640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c7058f4896c46729b8f3b2f878b7435: Generated new TSK 0
I20260806 06:54:46.079423 24640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:46.083199 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24563
2026-08-06T06:54:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:46 dist-test-slave-0kls krb5kdc[24662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:46 dist-test-slave-0kls krb5kdc[24662](info): set up 2 sockets
Aug 06 06:54:46 dist-test-slave-0kls krb5kdc[24662](info): commencing operation
krb5kdc: starting...
W20260806 06:54:47.109994 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:47 dist-test-slave-0kls krb5kdc[24662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:47.228690 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:47.309677 24675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:47.309940 24675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:47.309963 24675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:47.311085 24675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:47.311122 24675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:47.311136 24675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:47.311148 24675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:47.311160 24675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:47.312567 24675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:47.312778 24675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:47.313053 24675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:54:47.317991 24675 server_base.cc:1034] running on GCE node
W20260806 06:54:47.318001 24680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:47.318055 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:47.318017 24681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:47.318436 24675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:47.318764 24675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:47.319952 24675 hybrid_clock.cc:648] HybridClock initialized: now 1785999287319938 us; error 27 us; skew 500 ppm
Aug 06 06:54:47 dist-test-slave-0kls krb5kdc[24662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999287, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:47.322570 24675 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:47.323014 24675 webserver.cc:460] Webserver started at http://127.0.0.1:44931/ using document root <none> and password file <none>
I20260806 06:54:47.323244 24675 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:47.323338 24675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:47.323489 24675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:47.324555 24675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "62928efdc80d450f9472be0a4a236fe2"
format_stamp: "Formatted at 2026-08-06 06:54:47 on dist-test-slave-0kls"
server_key: "d2cf6dc456de7b0ee608c2b1430cfec0"
server_key_iv: "52c2e595b3c9c75df3eb402bd9fb167f"
server_key_version: "encryptionkey@0"
I20260806 06:54:47.324915 24675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "62928efdc80d450f9472be0a4a236fe2"
format_stamp: "Formatted at 2026-08-06 06:54:47 on dist-test-slave-0kls"
server_key: "d2cf6dc456de7b0ee608c2b1430cfec0"
server_key_iv: "52c2e595b3c9c75df3eb402bd9fb167f"
server_key_version: "encryptionkey@0"
I20260806 06:54:47.326342 24675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:47.327136 24690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:47.327355 24675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:47.327461 24675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "62928efdc80d450f9472be0a4a236fe2"
format_stamp: "Formatted at 2026-08-06 06:54:47 on dist-test-slave-0kls"
server_key: "d2cf6dc456de7b0ee608c2b1430cfec0"
server_key_iv: "52c2e595b3c9c75df3eb402bd9fb167f"
server_key_version: "encryptionkey@0"
I20260806 06:54:47.327535 24675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:47.343752 24675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:47.345719 24675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:47.345935 24675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:47.350020 24675 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39105
I20260806 06:54:47.350081 24742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39105 every 8 connection(s)
I20260806 06:54:47.350505 24675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:47.351508 24743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:47.353549 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24675
I20260806 06:54:47.353642 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:47.353890 4062 external_mini_cluster.cc:1428] Setting key f8e547ee7cf45124cc22e89b6926d4ea
I20260806 06:54:47.354007 24743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2: Bootstrap starting.
I20260806 06:54:47.354700 24743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:47.354997 24743 log.cc:824] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:47.355695 24743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2: No bootstrap required, opened a new log
I20260806 06:54:47.357388 24743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } }
I20260806 06:54:47.357545 24743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:47.357590 24743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62928efdc80d450f9472be0a4a236fe2, State: Initialized, Role: FOLLOWER
I20260806 06:54:47.357735 24743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } }
I20260806 06:54:47.357839 24743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:47.357882 24743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:47.357929 24743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:47.358525 24743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } }
I20260806 06:54:47.358640 24743 leader_election.cc:302] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62928efdc80d450f9472be0a4a236fe2; no voters:
I20260806 06:54:47.358805 24743 leader_election.cc:288] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:47.358840 24748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:47.358990 24748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [term 1 LEADER]: Becoming Leader. State: Replica: 62928efdc80d450f9472be0a4a236fe2, State: Running, Role: LEADER
I20260806 06:54:47.359125 24743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:47.359118 24748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } }
I20260806 06:54:47.359596 24749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62928efdc80d450f9472be0a4a236fe2. Latest consensus state: current_term: 1 leader_uuid: "62928efdc80d450f9472be0a4a236fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } } }
I20260806 06:54:47.359687 24749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:47.359786 24746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:47.354484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34741 (local address 127.0.0.1:39105)
0806 06:54:47.354677 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:47.354688 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:47.354720 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:54:47.355250 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:47.355257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:47.355336 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:47.355484 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:47.355701 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:47.356603 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:47.357386 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:47.357945 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:47.358069 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:47.358893 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:47.358897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:47.358897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:47.359107 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:47.359144 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:47.359161 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:47.359446 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:47.359484 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:54:47.359570 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260806 06:54:47.360265 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62928efdc80d450f9472be0a4a236fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62928efdc80d450f9472be0a4a236fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39105 } } }
I20260806 06:54:47.360352 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:47.360513 24755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:47.360977 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:47.361326 24755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:47.362923 24755 catalog_manager.cc:1347] Generated new cluster ID: aace9539133e4b0b9d58142f7c0c1ca7
I20260806 06:54:47.362970 24755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:47.368919 24746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:47.362347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40811 (local address 127.0.0.1:39105)
0806 06:54:47.362383 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:47.362385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:47.362600 (+ 215us) server_negotiation.cc:407] Connection header received
0806 06:54:47.362700 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:47.362701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:47.362712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:47.362768 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:47.365861 (+ 3093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:47.366602 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:47.367641 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:47.367930 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:47.367988 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:47.368681 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:47.368682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:47.368682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:47.368686 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:47.368700 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:47.368705 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:47.368754 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:47.368823 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:54:47.368845 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:54:47.371551 24755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:47.372144 24755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:47.374508 24755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62928efdc80d450f9472be0a4a236fe2: Generated new TSK 0
I20260806 06:54:47.374673 24755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:47.376849 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24675
2026-08-06T06:54:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:47 dist-test-slave-0kls krb5kdc[24774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:47 dist-test-slave-0kls krb5kdc[24774](info): set up 2 sockets
Aug 06 06:54:47 dist-test-slave-0kls krb5kdc[24774](info): commencing operation
krb5kdc: starting...
W20260806 06:54:48.398007 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:48 dist-test-slave-0kls krb5kdc[24774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:48.521209 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:48.604306 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:48.604578 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:48.604619 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:48.605911 24787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:48.605993 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:48.606020 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:48.606045 24787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:48.606076 24787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:48.607577 24787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:48.607892 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:48.608143 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:48.613494 24793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:48.613517 24795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:48.613517 24792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:48.613538 24787 server_base.cc:1034] running on GCE node
I20260806 06:54:48.613977 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:48.614240 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:48.615394 24787 hybrid_clock.cc:648] HybridClock initialized: now 1785999288615372 us; error 38 us; skew 500 ppm
Aug 06 06:54:48 dist-test-slave-0kls krb5kdc[24774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999288, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:48.622099 24787 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:48.622602 24787 webserver.cc:460] Webserver started at http://127.0.0.1:46861/ using document root <none> and password file <none>
I20260806 06:54:48.622844 24787 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:48.622892 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:48.623044 24787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:48.624157 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "e547e124b5344c05a88d20fd68dd0e8d"
format_stamp: "Formatted at 2026-08-06 06:54:48 on dist-test-slave-0kls"
server_key: "35e211d570848baba8431e2df183c6f8"
server_key_iv: "857456351c121e5d2e603863a996c106"
server_key_version: "encryptionkey@0"
I20260806 06:54:48.624562 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "e547e124b5344c05a88d20fd68dd0e8d"
format_stamp: "Formatted at 2026-08-06 06:54:48 on dist-test-slave-0kls"
server_key: "35e211d570848baba8431e2df183c6f8"
server_key_iv: "857456351c121e5d2e603863a996c106"
server_key_version: "encryptionkey@0"
I20260806 06:54:48.626087 24787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:54:48.626965 24802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:48.627167 24787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:48.627255 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "e547e124b5344c05a88d20fd68dd0e8d"
format_stamp: "Formatted at 2026-08-06 06:54:48 on dist-test-slave-0kls"
server_key: "35e211d570848baba8431e2df183c6f8"
server_key_iv: "857456351c121e5d2e603863a996c106"
server_key_version: "encryptionkey@0"
I20260806 06:54:48.627367 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:48.630831 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:48.633210 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:48.633381 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:48.637449 24787 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33873
I20260806 06:54:48.637480 24854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33873 every 8 connection(s)
I20260806 06:54:48.637879 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:48.638890 24855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:48.641185 24855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d: Bootstrap starting.
I20260806 06:54:48.641769 24855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:48.642168 24855 log.cc:824] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:48.642949 24855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d: No bootstrap required, opened a new log
I20260806 06:54:48.644721 24855 raft_consensus.cc:348] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } }
I20260806 06:54:48.644874 24855 raft_consensus.cc:374] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:48.644901 24855 raft_consensus.cc:729] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e547e124b5344c05a88d20fd68dd0e8d, State: Initialized, Role: FOLLOWER
I20260806 06:54:48.645095 24855 consensus_queue.cc:260] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } }
I20260806 06:54:48.645165 24855 raft_consensus.cc:388] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:48.645190 24855 raft_consensus.cc:482] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:48.645263 24855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:48.645956 24855 raft_consensus.cc:504] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } }
I20260806 06:54:48.646049 24855 leader_election.cc:302] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e547e124b5344c05a88d20fd68dd0e8d; no voters:
I20260806 06:54:48.646078 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24787
I20260806 06:54:48.646211 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:48.646270 24855 leader_election.cc:288] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:48.646301 24858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:48.646452 4062 external_mini_cluster.cc:1428] Setting key 1fc83bff5aaea18182693407dba9ecd2
I20260806 06:54:48.646483 24858 raft_consensus.cc:686] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [term 1 LEADER]: Becoming Leader. State: Replica: e547e124b5344c05a88d20fd68dd0e8d, State: Running, Role: LEADER
I20260806 06:54:48.646565 24858 consensus_queue.cc:237] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } }
I20260806 06:54:48.646644 24855 sys_catalog.cc:564] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:48.646891 24858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e547e124b5344c05a88d20fd68dd0e8d. Latest consensus state: current_term: 1 leader_uuid: "e547e124b5344c05a88d20fd68dd0e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } } }
I20260806 06:54:48.647025 24858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:48.647322 24864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:48.647712 24859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e547e124b5344c05a88d20fd68dd0e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547e124b5344c05a88d20fd68dd0e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33873 } } }
I20260806 06:54:48.647876 24859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:48.648010 24864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:48.650439 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:48.647353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46117 (local address 127.0.0.1:33873)
0806 06:54:48.648427 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:48.648436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:48.648474 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:54:48.648786 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:48.648796 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:48.648834 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:48.648976 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:48.649919 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:48.649924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:48.649924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:48.649929 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:48.649990 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:48.650006 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:48.650109 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:48.650180 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:54:48.650226 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":118,"threads_started":1}
I20260806 06:54:48.651474 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:48.654073 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:48.652988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48967 (local address 127.0.0.1:33873)
0806 06:54:48.653026 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:48.653027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:48.653104 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:54:48.653170 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:48.653170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:48.653174 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:48.653223 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:48.653765 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:48.653766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:48.653766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:48.653770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:48.653828 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:48.653834 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:48.653903 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:48.653972 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:54:48.654004 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:54:48.655263 24864 catalog_manager.cc:1347] Generated new cluster ID: c4b6f87b6c08435cbf870f0c15c0df35
I20260806 06:54:48.655318 24864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:48.667819 24864 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:48.668028 24864 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:48.677867 24864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e547e124b5344c05a88d20fd68dd0e8d: Generated new TSK 0
I20260806 06:54:48.678088 24864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:48.684753 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24787
2026-08-06T06:54:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:48 dist-test-slave-0kls krb5kdc[24886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:48 dist-test-slave-0kls krb5kdc[24886](info): set up 2 sockets
Aug 06 06:54:48 dist-test-slave-0kls krb5kdc[24886](info): commencing operation
krb5kdc: starting...
W20260806 06:54:49.722025 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:49 dist-test-slave-0kls krb5kdc[24886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:49.840884 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:49.922186 24899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:49.922441 24899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:49.922479 24899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:49.923659 24899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:49.923755 24899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:49.923780 24899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:49.923801 24899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:49.923830 24899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:49.925315 24899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:49.925594 24899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:49.925841 24899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:49.930755 24905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:49.930761 24904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:49.930816 24899 server_base.cc:1034] running on GCE node
W20260806 06:54:49.930733 24907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:49.931258 24899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:49.931576 24899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:49.932757 24899 hybrid_clock.cc:648] HybridClock initialized: now 1785999289932740 us; error 26 us; skew 500 ppm
Aug 06 06:54:49 dist-test-slave-0kls krb5kdc[24886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999289, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:49.935288 24899 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:49.935758 24899 webserver.cc:460] Webserver started at http://127.0.0.1:37405/ using document root <none> and password file <none>
I20260806 06:54:49.936003 24899 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:49.936071 24899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:49.936218 24899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:49.937278 24899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "38db1995e1dd471dab7255d9340e828e"
format_stamp: "Formatted at 2026-08-06 06:54:49 on dist-test-slave-0kls"
server_key: "8200902c79642c1694495fdbd8122aaa"
server_key_iv: "5683384d5da76cbce90a8ffacab7ffcf"
server_key_version: "encryptionkey@0"
I20260806 06:54:49.937638 24899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "38db1995e1dd471dab7255d9340e828e"
format_stamp: "Formatted at 2026-08-06 06:54:49 on dist-test-slave-0kls"
server_key: "8200902c79642c1694495fdbd8122aaa"
server_key_iv: "5683384d5da76cbce90a8ffacab7ffcf"
server_key_version: "encryptionkey@0"
I20260806 06:54:49.938987 24899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:54:49.939731 24914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:49.939900 24899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:49.939991 24899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "38db1995e1dd471dab7255d9340e828e"
format_stamp: "Formatted at 2026-08-06 06:54:49 on dist-test-slave-0kls"
server_key: "8200902c79642c1694495fdbd8122aaa"
server_key_iv: "5683384d5da76cbce90a8ffacab7ffcf"
server_key_version: "encryptionkey@0"
I20260806 06:54:49.940076 24899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:49.949901 24899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:49.951766 24899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:49.951927 24899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:49.956211 24899 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44339
I20260806 06:54:49.956264 24966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44339 every 8 connection(s)
I20260806 06:54:49.956641 24899 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:49.957617 24967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:49.959991 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e: Bootstrap starting.
I20260806 06:54:49.960534 24967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:49.960764 24967 log.cc:824] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:49.961411 24967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e: No bootstrap required, opened a new log
I20260806 06:54:49.962963 24967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } }
I20260806 06:54:49.963102 24967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:49.963125 24967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38db1995e1dd471dab7255d9340e828e, State: Initialized, Role: FOLLOWER
I20260806 06:54:49.963271 24967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } }
I20260806 06:54:49.963352 24967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:49.963375 24967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:49.963405 24967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:49.963891 24967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } }
I20260806 06:54:49.963984 24967 leader_election.cc:302] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38db1995e1dd471dab7255d9340e828e; no voters:
I20260806 06:54:49.964108 24967 leader_election.cc:288] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:49.964165 24970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:49.964332 24970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [term 1 LEADER]: Becoming Leader. State: Replica: 38db1995e1dd471dab7255d9340e828e, State: Running, Role: LEADER
I20260806 06:54:49.964424 24967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:49.964461 24970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } }
I20260806 06:54:49.964910 24972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38db1995e1dd471dab7255d9340e828e. Latest consensus state: current_term: 1 leader_uuid: "38db1995e1dd471dab7255d9340e828e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } } }
I20260806 06:54:49.965057 24971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38db1995e1dd471dab7255d9340e828e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38db1995e1dd471dab7255d9340e828e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44339 } } }
I20260806 06:54:49.965150 24971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:49.965047 24972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:49.965837 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 24899
I20260806 06:54:49.965950 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:49.966104 4062 external_mini_cluster.cc:1428] Setting key a82aba06534e063cbe6375f1f2380080
I20260806 06:54:49.965548 24979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:49.966758 24979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:49.968406 24979 catalog_manager.cc:1347] Generated new cluster ID: 9082141436eb4e5eb7f9c624247bad2c
I20260806 06:54:49.968461 24979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:54:49 dist-test-slave-0kls krb5kdc[24886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:54:49.974560 24979 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:49.975235 24979 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:49.977003 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:49.966736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:40705 (local address 127.0.0.1:44339)
0806 06:54:49.966925 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:49.966934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:49.966963 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:54:49.967081 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:49.967088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:49.967145 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:49.967278 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:49.968078 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:49.969188 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:49.970029 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:49.970468 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:49.970591 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:49.972996 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:49.973000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:49.973001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:49.973013 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:49.975239 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:49.975766 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:49.975769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:49.975770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:49.975875 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:49.976120 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:49.976121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:49.976121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:49.976385 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:49.976548 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:49.976681 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:54:49.976780 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260806 06:54:49.978030 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:49.979219 24979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38db1995e1dd471dab7255d9340e828e: Generated new TSK 0
I20260806 06:54:49.979358 24979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:49.985994 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:49.979566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:32839 (local address 127.0.0.1:44339)
0806 06:54:49.979626 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:49.979627 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:49.979773 (+ 146us) server_negotiation.cc:407] Connection header received
0806 06:54:49.979848 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:49.979849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:49.979859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:49.979915 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:49.980450 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:49.980982 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:49.981712 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:49.982011 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:49.982091 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:49.983327 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:49.983328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:49.983329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:49.983337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:49.984780 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:49.985160 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:49.985161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:49.985161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:49.985222 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:49.985455 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:49.985456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:49.985457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:49.985567 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:49.985693 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:49.985830 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 06:54:49.985893 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:54:49.988314 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 24899
W20260806 06:54:49.992120 24884 connection.cc:535] client connection to 127.0.0.1:44339 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44339): Connection reset by peer (error 104)
2026-08-06T06:54:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:50 dist-test-slave-0kls krb5kdc[24998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:50 dist-test-slave-0kls krb5kdc[24998](info): set up 2 sockets
Aug 06 06:54:50 dist-test-slave-0kls krb5kdc[24998](info): commencing operation
krb5kdc: starting...
W20260806 06:54:51.022068 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:51 dist-test-slave-0kls krb5kdc[24998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:51.140918 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:51.221897 25011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:51.222150 25011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:51.222188 25011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:51.223330 25011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:51.223400 25011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:51.223433 25011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:51.223462 25011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:51.223485 25011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:51.224937 25011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:51.225211 25011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:51.225445 25011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:51.230386 25017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:51.230417 25016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:51.230623 25019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:51.230633 25011 server_base.cc:1034] running on GCE node
I20260806 06:54:51.230886 25011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:51.231177 25011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:51.232369 25011 hybrid_clock.cc:648] HybridClock initialized: now 1785999291232362 us; error 35 us; skew 500 ppm
Aug 06 06:54:51 dist-test-slave-0kls krb5kdc[24998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999291, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:51.234853 25011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:51.235327 25011 webserver.cc:460] Webserver started at http://127.0.0.1:41933/ using document root <none> and password file <none>
I20260806 06:54:51.235559 25011 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:51.235632 25011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:51.235777 25011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:51.236888 25011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "3ddb91f724a4422bbe1391c9a46db557"
format_stamp: "Formatted at 2026-08-06 06:54:51 on dist-test-slave-0kls"
server_key: "d3194218251c9879c7ed90ef42c25acc"
server_key_iv: "55d01038cb2393342695e31c4b4eaa1a"
server_key_version: "encryptionkey@0"
I20260806 06:54:51.237250 25011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "3ddb91f724a4422bbe1391c9a46db557"
format_stamp: "Formatted at 2026-08-06 06:54:51 on dist-test-slave-0kls"
server_key: "d3194218251c9879c7ed90ef42c25acc"
server_key_iv: "55d01038cb2393342695e31c4b4eaa1a"
server_key_version: "encryptionkey@0"
I20260806 06:54:51.238639 25011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:54:51.239413 25026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:51.239583 25011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:51.239681 25011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "3ddb91f724a4422bbe1391c9a46db557"
format_stamp: "Formatted at 2026-08-06 06:54:51 on dist-test-slave-0kls"
server_key: "d3194218251c9879c7ed90ef42c25acc"
server_key_iv: "55d01038cb2393342695e31c4b4eaa1a"
server_key_version: "encryptionkey@0"
I20260806 06:54:51.239758 25011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:51.246925 25011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:51.248926 25011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:51.249076 25011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:51.252943 25011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35505
I20260806 06:54:51.252974 25078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35505 every 8 connection(s)
I20260806 06:54:51.253326 25011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:51.254354 25079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:51.255921 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25011
I20260806 06:54:51.256052 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:51.256256 4062 external_mini_cluster.cc:1428] Setting key f93368320f36b253edc7bac568e870e6
I20260806 06:54:51.256949 25079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557: Bootstrap starting.
I20260806 06:54:51.257530 25079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:51.257869 25079 log.cc:824] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:51.258533 25079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557: No bootstrap required, opened a new log
I20260806 06:54:51.260222 25079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } }
I20260806 06:54:51.260385 25079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:51.260434 25079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ddb91f724a4422bbe1391c9a46db557, State: Initialized, Role: FOLLOWER
I20260806 06:54:51.260565 25079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } }
I20260806 06:54:51.260655 25079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:51.260687 25079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:51.260731 25079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:51.261241 25079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } }
I20260806 06:54:51.261384 25079 leader_election.cc:302] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ddb91f724a4422bbe1391c9a46db557; no voters:
I20260806 06:54:51.261600 25084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:51.261576 25079 leader_election.cc:288] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:51.261746 25084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [term 1 LEADER]: Becoming Leader. State: Replica: 3ddb91f724a4422bbe1391c9a46db557, State: Running, Role: LEADER
I20260806 06:54:51.261826 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:51.256776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:54311 (local address 127.0.0.1:35505)
0806 06:54:51.256962 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:51.256975 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:51.257020 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:54:51.257474 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:51.257481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:51.257534 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:51.257691 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:51.257908 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:51.258882 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:51.259653 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:51.260062 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:51.260189 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:51.260912 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:51.260916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:51.260916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:51.261091 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:51.261119 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:51.261129 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:51.261221 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:51.261369 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:54:51.261510 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260806 06:54:51.261861 25084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } }
I20260806 06:54:51.261900 25079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:51.262153 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddb91f724a4422bbe1391c9a46db557. Latest consensus state: current_term: 1 leader_uuid: "3ddb91f724a4422bbe1391c9a46db557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } } }
I20260806 06:54:51.262249 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:51.262398 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ddb91f724a4422bbe1391c9a46db557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddb91f724a4422bbe1391c9a46db557" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35505 } } }
I20260806 06:54:51.262454 25089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:51.262470 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:51.262823 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:51.263084 25089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:51.264657 25089 catalog_manager.cc:1347] Generated new cluster ID: 6623b73cb9e7475485cd999db24933cf
I20260806 06:54:51.264706 25089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:51.270576 25089 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:51.271109 25089 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:51.273995 25082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:51.264415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:47173 (local address 127.0.0.1:35505)
0806 06:54:51.264447 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:51.264449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:51.270436 (+ 5987us) server_negotiation.cc:407] Connection header received
0806 06:54:51.270515 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:51.270517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:51.270528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:51.270590 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:51.270770 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:51.271541 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:51.272533 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:51.272851 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:51.272923 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:51.273674 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:51.273676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:51.273676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:51.273681 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:51.273696 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:51.273701 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:51.273765 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:51.273860 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:54:51.273873 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:54:51.274624 25089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ddb91f724a4422bbe1391c9a46db557: Generated new TSK 0
I20260806 06:54:51.274787 25089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:51.281744 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25011
W20260806 06:54:51.286190 24996 connection.cc:535] client connection to 127.0.0.1:35505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35505): Connection reset by peer (error 104)
2026-08-06T06:54:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:51 dist-test-slave-0kls krb5kdc[25110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:51 dist-test-slave-0kls krb5kdc[25110](info): set up 2 sockets
Aug 06 06:54:51 dist-test-slave-0kls krb5kdc[25110](info): commencing operation
krb5kdc: starting...
W20260806 06:54:52.301987 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:52 dist-test-slave-0kls krb5kdc[25110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:52.418174 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:52.499403 25123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:52.499615 25123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:52.499640 25123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:52.500819 25123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:52.500869 25123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:52.500885 25123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:52.500897 25123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:52.500909 25123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:52.502501 25123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:52.502763 25123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:52.502987 25123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:52.507828 25131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:52.507843 25129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:52.507881 25128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:52.508018 25123 server_base.cc:1034] running on GCE node
I20260806 06:54:52.508244 25123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:52.508545 25123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:52.509692 25123 hybrid_clock.cc:648] HybridClock initialized: now 1785999292509674 us; error 35 us; skew 500 ppm
Aug 06 06:54:52 dist-test-slave-0kls krb5kdc[25110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999292, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:52.512207 25123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:52.512707 25123 webserver.cc:460] Webserver started at http://127.0.0.1:33679/ using document root <none> and password file <none>
I20260806 06:54:52.512964 25123 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:52.513011 25123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:52.513149 25123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:52.514314 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "2a70afa02abc4ab9a6543d5af8fc7540"
format_stamp: "Formatted at 2026-08-06 06:54:52 on dist-test-slave-0kls"
server_key: "8e5659d4e039db63736f6db4527b5cd9"
server_key_iv: "cb923ecc150d0de74377e0b2ca95c44c"
server_key_version: "encryptionkey@0"
I20260806 06:54:52.514694 25123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "2a70afa02abc4ab9a6543d5af8fc7540"
format_stamp: "Formatted at 2026-08-06 06:54:52 on dist-test-slave-0kls"
server_key: "8e5659d4e039db63736f6db4527b5cd9"
server_key_iv: "cb923ecc150d0de74377e0b2ca95c44c"
server_key_version: "encryptionkey@0"
I20260806 06:54:52.516076 25123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:52.516875 25138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:52.517084 25123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:54:52.517170 25123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "2a70afa02abc4ab9a6543d5af8fc7540"
format_stamp: "Formatted at 2026-08-06 06:54:52 on dist-test-slave-0kls"
server_key: "8e5659d4e039db63736f6db4527b5cd9"
server_key_iv: "cb923ecc150d0de74377e0b2ca95c44c"
server_key_version: "encryptionkey@0"
I20260806 06:54:52.517246 25123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:52.520690 25123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:52.523057 25123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:52.523233 25123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:52.527192 25123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33709
I20260806 06:54:52.527242 25190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33709 every 8 connection(s)
I20260806 06:54:52.527599 25123 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:52.528601 25191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:52.531001 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: Bootstrap starting.
I20260806 06:54:52.531560 25191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:52.531796 25191 log.cc:824] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:52.532545 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: No bootstrap required, opened a new log
I20260806 06:54:52.532711 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25123
I20260806 06:54:52.532828 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:52.533005 4062 external_mini_cluster.cc:1428] Setting key a47c73feca13f1495945479e785176f3
I20260806 06:54:52.534744 25191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260806 06:54:52.534929 25191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:52.534974 25191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a70afa02abc4ab9a6543d5af8fc7540, State: Initialized, Role: FOLLOWER
I20260806 06:54:52.535116 25191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260806 06:54:52.535223 25191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:52.535249 25191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:52.535333 25191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:52.536159 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:52.533551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:43455 (local address 127.0.0.1:33709)
0806 06:54:52.533737 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:52.533745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:52.533772 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:54:52.534722 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:52.534729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:52.534759 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:52.534879 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:52.535616 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:52.535621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:52.535621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:52.535625 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:52.535671 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:52.535685 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:52.535763 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:52.535840 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:54:52.535895 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 06:54:52.536568 25191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260806 06:54:52.536676 25191 leader_election.cc:302] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a70afa02abc4ab9a6543d5af8fc7540; no voters:
I20260806 06:54:52.536835 25191 leader_election.cc:288] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:52.536870 25196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:52.537073 25196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [term 1 LEADER]: Becoming Leader. State: Replica: 2a70afa02abc4ab9a6543d5af8fc7540, State: Running, Role: LEADER
I20260806 06:54:52.537091 25191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:52.537160 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:52.537164 25196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } }
I20260806 06:54:52.537477 25196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } } }
I20260806 06:54:52.537570 25196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:52.537705 25196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a70afa02abc4ab9a6543d5af8fc7540. Latest consensus state: current_term: 1 leader_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a70afa02abc4ab9a6543d5af8fc7540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33709 } } }
I20260806 06:54:52.537767 25196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540 [sys.catalog]: This master's current role is: LEADER
W20260806 06:54:52.538875 25214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:54:52.538944 25214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:54:52.539012 25204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:52.539175 25204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:52.540114 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:52.538763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:51327 (local address 127.0.0.1:33709)
0806 06:54:52.538800 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:52.538803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:52.539047 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:54:52.539140 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:52.539141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:52.539146 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:52.539174 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:52.539876 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:52.539877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:52.539878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:52.539882 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:52.539897 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:52.539901 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:52.539931 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:52.540043 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:54:52.540059 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:52.540782 25204 catalog_manager.cc:1347] Generated new cluster ID: 8647e36788184f5781a14b29f2245f73
I20260806 06:54:52.540832 25204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:52.552487 25204 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:52.552682 25204 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:52.562038 25204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a70afa02abc4ab9a6543d5af8fc7540: Generated new TSK 0
I20260806 06:54:52.562281 25204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:52.564589 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25123
2026-08-06T06:54:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:52 dist-test-slave-0kls krb5kdc[25222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:52 dist-test-slave-0kls krb5kdc[25222](info): set up 2 sockets
Aug 06 06:54:52 dist-test-slave-0kls krb5kdc[25222](info): commencing operation
krb5kdc: starting...
W20260806 06:54:53.590000 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:53.709126 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:53.791268 25235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:53.791478 25235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:53.791501 25235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:53.792570 25235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:53.792604 25235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:53.792619 25235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:53.792632 25235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:53.792644 25235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:53.794117 25235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:53.794315 25235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:53.794487 25235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:53.799233 25241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:53.799273 25243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:53.799242 25240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:53.799536 25235 server_base.cc:1034] running on GCE node
I20260806 06:54:53.799757 25235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:53.800007 25235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:53.801148 25235 hybrid_clock.cc:648] HybridClock initialized: now 1785999293801140 us; error 33 us; skew 500 ppm
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999293, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:53.809762 25235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:53.810315 25235 webserver.cc:460] Webserver started at http://127.0.0.1:43725/ using document root <none> and password file <none>
I20260806 06:54:53.810519 25235 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:53.810562 25235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:53.810655 25235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:53.811715 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "2dc5de4dfee54fcba341af5cbd611b94"
format_stamp: "Formatted at 2026-08-06 06:54:53 on dist-test-slave-0kls"
server_key: "4bd794f1f447f418a2918e739dfe424f"
server_key_iv: "36fc9be8e43a33d24dab9ab1b1f30b68"
server_key_version: "encryptionkey@0"
I20260806 06:54:53.812019 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "2dc5de4dfee54fcba341af5cbd611b94"
format_stamp: "Formatted at 2026-08-06 06:54:53 on dist-test-slave-0kls"
server_key: "4bd794f1f447f418a2918e739dfe424f"
server_key_iv: "36fc9be8e43a33d24dab9ab1b1f30b68"
server_key_version: "encryptionkey@0"
I20260806 06:54:53.813373 25235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:54:53.814270 25250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:53.814527 25235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:54:53.814651 25235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "2dc5de4dfee54fcba341af5cbd611b94"
format_stamp: "Formatted at 2026-08-06 06:54:53 on dist-test-slave-0kls"
server_key: "4bd794f1f447f418a2918e739dfe424f"
server_key_iv: "36fc9be8e43a33d24dab9ab1b1f30b68"
server_key_version: "encryptionkey@0"
I20260806 06:54:53.814745 25235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:53.832693 25235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:53.835067 25235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:53.835251 25235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:53.839660 25235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45099
I20260806 06:54:53.839710 25302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45099 every 8 connection(s)
I20260806 06:54:53.840109 25235 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:53.841099 25303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:53.843586 25303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94: Bootstrap starting.
I20260806 06:54:53.844107 25303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:53.844180 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25235
I20260806 06:54:53.844321 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:53.844498 4062 external_mini_cluster.cc:1428] Setting key 61fdbedbde6dde3288bba459b7d46865
I20260806 06:54:53.844503 25303 log.cc:824] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:53.845538 25303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94: No bootstrap required, opened a new log
I20260806 06:54:53.848094 25303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } }
I20260806 06:54:53.848281 25303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:53.848315 25303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc5de4dfee54fcba341af5cbd611b94, State: Initialized, Role: FOLLOWER
I20260806 06:54:53.848438 25303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } }
I20260806 06:54:53.848531 25303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:53.848567 25303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:53.848616 25303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:53.849179 25303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } }
I20260806 06:54:53.849311 25303 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dc5de4dfee54fcba341af5cbd611b94; no voters:
I20260806 06:54:53.849486 25303 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:53.849555 25308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:53.849723 25303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:53.849730 25308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [term 1 LEADER]: Becoming Leader. State: Replica: 2dc5de4dfee54fcba341af5cbd611b94, State: Running, Role: LEADER
I20260806 06:54:53.849967 25308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } }
I20260806 06:54:53.850414 25309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dc5de4dfee54fcba341af5cbd611b94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } } }
I20260806 06:54:53.851061 25309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:53.850577 25310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dc5de4dfee54fcba341af5cbd611b94. Latest consensus state: current_term: 1 leader_uuid: "2dc5de4dfee54fcba341af5cbd611b94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc5de4dfee54fcba341af5cbd611b94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45099 } } }
I20260806 06:54:53.851373 25324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:53.851562 25310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:53.851979 25324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:54:53.853644 25324 catalog_manager.cc:1347] Generated new cluster ID: 7e6fd28f77394d319eed6243516fd467
I20260806 06:54:53.853693 25324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:53.861292 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:53.845070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:43767 (local address 127.0.0.1:45099)
0806 06:54:53.845249 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:53.845258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:53.845291 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:54:53.845934 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:53.845944 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:53.846014 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:53.846182 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:53.846831 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:53.848430 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:53.849300 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:53.849942 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:53.850101 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:53.853247 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:53.853251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:53.853251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:53.853263 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:53.859463 (+ 6200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:53.859984 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:53.859987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:53.859987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:53.860095 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:53.860362 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:53.860363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:53.860363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:53.860670 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:53.860819 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:53.860972 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:54:53.861068 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260806 06:54:53.862308 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:53.868291 25324 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:53.869009 25324 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:53.871166 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:53.863795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:48365 (local address 127.0.0.1:45099)
0806 06:54:53.863832 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:53.863834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:53.863968 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:54:53.864024 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:53.864025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:53.864035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:53.864084 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:53.864696 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:53.865438 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:53.866497 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:53.866811 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:53.866878 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:53.868276 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:53.868277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:53.868278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:53.868285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:53.869927 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:53.870354 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:53.870355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:53.870356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:53.870412 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:53.870644 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:53.870644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:53.870645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:53.870757 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:53.870850 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:53.871015 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:54:53.871079 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:54:53.874452 25324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dc5de4dfee54fcba341af5cbd611b94: Generated new TSK 0
I20260806 06:54:53.874639 25324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:53.879511 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25235
W20260806 06:54:53.883925 25220 connection.cc:535] client connection to 127.0.0.1:45099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45099): Connection reset by peer (error 104)
2026-08-06T06:54:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25334](info): set up 2 sockets
Aug 06 06:54:53 dist-test-slave-0kls krb5kdc[25334](info): commencing operation
krb5kdc: starting...
W20260806 06:54:54.938102 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:55 dist-test-slave-0kls krb5kdc[25334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:55.056212 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:55.137890 25347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:55.138121 25347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:55.138159 25347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:55.139334 25347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:55.139405 25347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:55.139429 25347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:55.139458 25347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:55.139485 25347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:55.140983 25347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:55.141258 25347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:55.141485 25347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:55.146377 25353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:55.146404 25355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:55.146469 25352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:55.146495 25347 server_base.cc:1034] running on GCE node
I20260806 06:54:55.146839 25347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:55.147140 25347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:55.148303 25347 hybrid_clock.cc:648] HybridClock initialized: now 1785999295148285 us; error 37 us; skew 500 ppm
Aug 06 06:54:55 dist-test-slave-0kls krb5kdc[25334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999295, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:55.150852 25347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:55.151299 25347 webserver.cc:460] Webserver started at http://127.0.0.1:38459/ using document root <none> and password file <none>
I20260806 06:54:55.151501 25347 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:55.151541 25347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:55.151628 25347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:55.152642 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "f58c5c7ced3f41438de6a03d65586821"
format_stamp: "Formatted at 2026-08-06 06:54:55 on dist-test-slave-0kls"
server_key: "b8fc69ee1a703575c95bd1c361b17c3d"
server_key_iv: "b0bbe7aad7dfca409d8ea9c5624451b0"
server_key_version: "encryptionkey@0"
I20260806 06:54:55.152942 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "f58c5c7ced3f41438de6a03d65586821"
format_stamp: "Formatted at 2026-08-06 06:54:55 on dist-test-slave-0kls"
server_key: "b8fc69ee1a703575c95bd1c361b17c3d"
server_key_iv: "b0bbe7aad7dfca409d8ea9c5624451b0"
server_key_version: "encryptionkey@0"
I20260806 06:54:55.154382 25347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:55.155011 25362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:55.155184 25347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:54:55.155282 25347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "f58c5c7ced3f41438de6a03d65586821"
format_stamp: "Formatted at 2026-08-06 06:54:55 on dist-test-slave-0kls"
server_key: "b8fc69ee1a703575c95bd1c361b17c3d"
server_key_iv: "b0bbe7aad7dfca409d8ea9c5624451b0"
server_key_version: "encryptionkey@0"
I20260806 06:54:55.155349 25347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:55.168609 25347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:55.170665 25347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:55.170817 25347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:55.175012 25347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43505
I20260806 06:54:55.175052 25414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43505 every 8 connection(s)
I20260806 06:54:55.175449 25347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:55.176342 25415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:55.178820 25415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821: Bootstrap starting.
I20260806 06:54:55.179417 25415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:55.179697 25415 log.cc:824] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:55.180361 25415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821: No bootstrap required, opened a new log
I20260806 06:54:55.180927 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25347
I20260806 06:54:55.181048 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:55.181229 4062 external_mini_cluster.cc:1428] Setting key 92d643c4305a1f5fe371fbe94b9b5617
I20260806 06:54:55.182436 25415 raft_consensus.cc:348] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } }
I20260806 06:54:55.182592 25415 raft_consensus.cc:374] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:55.182631 25415 raft_consensus.cc:729] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58c5c7ced3f41438de6a03d65586821, State: Initialized, Role: FOLLOWER
I20260806 06:54:55.182824 25415 consensus_queue.cc:260] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } }
I20260806 06:54:55.182907 25415 raft_consensus.cc:388] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:55.182930 25415 raft_consensus.cc:482] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:55.182998 25415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:55.183542 25415 raft_consensus.cc:504] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } }
I20260806 06:54:55.183624 25415 leader_election.cc:302] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f58c5c7ced3f41438de6a03d65586821; no voters:
I20260806 06:54:55.183802 25415 leader_election.cc:288] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:55.183851 25420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:55.184022 25420 raft_consensus.cc:686] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [term 1 LEADER]: Becoming Leader. State: Replica: f58c5c7ced3f41438de6a03d65586821, State: Running, Role: LEADER
I20260806 06:54:55.184074 25415 sys_catalog.cc:564] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:55.184123 25420 consensus_queue.cc:237] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } }
I20260806 06:54:55.184517 25420 sys_catalog.cc:455] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f58c5c7ced3f41438de6a03d65586821. Latest consensus state: current_term: 1 leader_uuid: "f58c5c7ced3f41438de6a03d65586821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } } }
I20260806 06:54:55.184644 25420 sys_catalog.cc:458] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:55.184928 25429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:55.185415 25429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:55.185909 25421 sys_catalog.cc:455] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f58c5c7ced3f41438de6a03d65586821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58c5c7ced3f41438de6a03d65586821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43505 } } }
I20260806 06:54:55.185999 25421 sys_catalog.cc:458] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:55.187057 25429 catalog_manager.cc:1347] Generated new cluster ID: f718eda9a75e45659a7540b16401a26d
I20260806 06:54:55.187112 25429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:55.187713 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:55.181731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:45345 (local address 127.0.0.1:43505)
0806 06:54:55.181940 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:55.181947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:55.181972 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:54:55.182417 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:55.182424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:55.182474 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:55.182602 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:55.182844 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:55.184363 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:55.185568 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:55.186174 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:55.186280 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:55.187009 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:55.187013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:55.187014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:55.187245 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:55.187283 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:55.187299 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:55.187381 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:55.187481 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:54:55.187508 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260806 06:54:55.188670 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:55.193702 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:55.190309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:34153 (local address 127.0.0.1:43505)
0806 06:54:55.190338 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:55.190340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:55.190347 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:54:55.190417 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:55.190418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:55.190428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:55.190514 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:55.190649 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:55.191344 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:55.192346 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:55.192674 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:55.192730 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:55.193444 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:55.193446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:55.193446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:55.193450 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:55.193465 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:55.193469 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:55.193525 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:55.193615 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:54:55.193627 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 06:54:55.209615 25429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:55.210266 25429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:55.213702 25429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f58c5c7ced3f41438de6a03d65586821: Generated new TSK 0
I20260806 06:54:55.213930 25429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:55.222869 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25347
W20260806 06:54:55.227214 25332 connection.cc:535] client connection to 127.0.0.1:43505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43505): Connection reset by peer (error 104)
2026-08-06T06:54:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:55 dist-test-slave-0kls krb5kdc[25446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:55 dist-test-slave-0kls krb5kdc[25446](info): set up 2 sockets
Aug 06 06:54:55 dist-test-slave-0kls krb5kdc[25446](info): commencing operation
krb5kdc: starting...
W20260806 06:54:56.246194 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:56 dist-test-slave-0kls krb5kdc[25446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:54:56.360698 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:56.441901 25459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:56.442134 25459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:56.442174 25459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:56.443328 25459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:56.443392 25459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:56.443414 25459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:56.443444 25459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:56.443472 25459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:56.444933 25459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46395
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:56.445209 25459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:56.445441 25459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:56.450142 25465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:56.450174 25464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:56.450277 25467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:56.450340 25459 server_base.cc:1034] running on GCE node
I20260806 06:54:56.450675 25459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:56.450979 25459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:56.452135 25459 hybrid_clock.cc:648] HybridClock initialized: now 1785999296452126 us; error 34 us; skew 500 ppm
Aug 06 06:54:56 dist-test-slave-0kls krb5kdc[25446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999296, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:56.454581 25459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:54:56.455062 25459 webserver.cc:460] Webserver started at http://127.0.0.1:41809/ using document root <none> and password file <none>
I20260806 06:54:56.455288 25459 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:56.455353 25459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:56.455495 25459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:56.456557 25459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "1a80acdedfec48a79b6d989594c3e7a3"
format_stamp: "Formatted at 2026-08-06 06:54:56 on dist-test-slave-0kls"
server_key: "7d803bb339f33c853fbc3456f74411a5"
server_key_iv: "3f35371e15f24a1a4c5927fff1db406b"
server_key_version: "encryptionkey@0"
I20260806 06:54:56.456918 25459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "1a80acdedfec48a79b6d989594c3e7a3"
format_stamp: "Formatted at 2026-08-06 06:54:56 on dist-test-slave-0kls"
server_key: "7d803bb339f33c853fbc3456f74411a5"
server_key_iv: "3f35371e15f24a1a4c5927fff1db406b"
server_key_version: "encryptionkey@0"
I20260806 06:54:56.458317 25459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:54:56.459129 25474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:56.459306 25459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:54:56.459405 25459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "1a80acdedfec48a79b6d989594c3e7a3"
format_stamp: "Formatted at 2026-08-06 06:54:56 on dist-test-slave-0kls"
server_key: "7d803bb339f33c853fbc3456f74411a5"
server_key_iv: "3f35371e15f24a1a4c5927fff1db406b"
server_key_version: "encryptionkey@0"
I20260806 06:54:56.459477 25459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:56.462808 25459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:56.464699 25459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:56.464854 25459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:56.468856 25459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46395
I20260806 06:54:56.468888 25526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46395 every 8 connection(s)
I20260806 06:54:56.469226 25459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:56.470256 25527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:56.472447 25527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3: Bootstrap starting.
I20260806 06:54:56.472976 25527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:56.473227 25527 log.cc:824] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:56.474002 25527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3: No bootstrap required, opened a new log
I20260806 06:54:56.475437 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25459
I20260806 06:54:56.475558 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:56.475728 4062 external_mini_cluster.cc:1428] Setting key 57aa119913d916af15961e7cdd6e3b8f
I20260806 06:54:56.475751 25527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 06:54:56.475939 25527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:56.475970 25527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a80acdedfec48a79b6d989594c3e7a3, State: Initialized, Role: FOLLOWER
I20260806 06:54:56.476133 25527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 06:54:56.476225 25527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:56.476284 25527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:56.476380 25527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:56.477141 25527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 06:54:56.477255 25527 leader_election.cc:302] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a80acdedfec48a79b6d989594c3e7a3; no voters:
I20260806 06:54:56.477451 25527 leader_election.cc:288] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:56.477520 25532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:56.477727 25532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1a80acdedfec48a79b6d989594c3e7a3, State: Running, Role: LEADER
I20260806 06:54:56.477749 25527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:56.477916 25532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260806 06:54:56.478274 25534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a80acdedfec48a79b6d989594c3e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260806 06:54:56.478397 25534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:56.478632 25541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:56.478827 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:56.476230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:54869 (local address 127.0.0.1:46395)
0806 06:54:56.476607 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:56.476617 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:56.476648 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:54:56.477054 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:56.477063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:56.477100 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:56.477258 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:56.478127 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:56.478131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:56.478132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:56.478136 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:56.478184 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:56.478199 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:56.478541 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:56.478575 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:54:56.478629 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":93,"threads_started":1}
I20260806 06:54:56.479161 25541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:56.479418 25533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a80acdedfec48a79b6d989594c3e7a3. Latest consensus state: current_term: 1 leader_uuid: "1a80acdedfec48a79b6d989594c3e7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a80acdedfec48a79b6d989594c3e7a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260806 06:54:56.479605 25533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:56.480005 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:56.482828 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:56.481584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:59011 (local address 127.0.0.1:46395)
0806 06:54:56.481619 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:56.481621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:56.481943 (+ 322us) server_negotiation.cc:407] Connection header received
0806 06:54:56.482006 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:56.482007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:56.482013 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:56.482082 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:56.482582 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:56.482583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:56.482584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:56.482587 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:56.482599 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:56.482603 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:56.482642 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:56.482703 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:54:56.482713 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:56.485818 25541 catalog_manager.cc:1347] Generated new cluster ID: 3052cfa9670f45ed869aaeb622ec468d
I20260806 06:54:56.485874 25541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:56.498380 25541 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:56.498553 25541 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:56.505215 25541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a80acdedfec48a79b6d989594c3e7a3: Generated new TSK 0
I20260806 06:54:56.505416 25541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:56.515329 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25459
2026-08-06T06:54:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:56 dist-test-slave-0kls krb5kdc[25558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:56 dist-test-slave-0kls krb5kdc[25558](info): set up 2 sockets
Aug 06 06:54:56 dist-test-slave-0kls krb5kdc[25558](info): commencing operation
krb5kdc: starting...
W20260806 06:54:57.546036 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:54:57.662600 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39037
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:57.743641 25571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:57.743876 25571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:57.743901 25571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:57.745167 25571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:57.745209 25571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:57.745224 25571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:57.745237 25571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:57.745266 25571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:57.746922 25571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:39037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:39037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:57.747174 25571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:57.747404 25571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:57.752372 25577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:57.752347 25579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:57.752408 25576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:57.752413 25571 server_base.cc:1034] running on GCE node
I20260806 06:54:57.752771 25571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:57.753078 25571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:57.754213 25571 hybrid_clock.cc:648] HybridClock initialized: now 1785999297754193 us; error 38 us; skew 500 ppm
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:57.756690 25571 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:57.757154 25571 webserver.cc:460] Webserver started at http://127.3.247.190:33141/ using document root <none> and password file <none>
I20260806 06:54:57.757350 25571 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:57.757393 25571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:57.757481 25571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:57.758526 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "704f39e825814c0da4ae5f681d972250"
format_stamp: "Formatted at 2026-08-06 06:54:57 on dist-test-slave-0kls"
server_key: "8184c37edcfe854e99732be20cfb0369"
server_key_iv: "59d1f5ec68e83eab8b4badd4eaed42d3"
server_key_version: "encryptionkey@0"
I20260806 06:54:57.758821 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "704f39e825814c0da4ae5f681d972250"
format_stamp: "Formatted at 2026-08-06 06:54:57 on dist-test-slave-0kls"
server_key: "8184c37edcfe854e99732be20cfb0369"
server_key_iv: "59d1f5ec68e83eab8b4badd4eaed42d3"
server_key_version: "encryptionkey@0"
I20260806 06:54:57.760033 25571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:54:57.760694 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:57.760859 25571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:57.760958 25571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "704f39e825814c0da4ae5f681d972250"
format_stamp: "Formatted at 2026-08-06 06:54:57 on dist-test-slave-0kls"
server_key: "8184c37edcfe854e99732be20cfb0369"
server_key_iv: "59d1f5ec68e83eab8b4badd4eaed42d3"
server_key_version: "encryptionkey@0"
I20260806 06:54:57.761030 25571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:57.782274 25571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:57.784677 25571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:57.784842 25571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:57.789072 25571 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:39037
I20260806 06:54:57.789131 25638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:39037 every 8 connection(s)
I20260806 06:54:57.789486 25571 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:57.790397 25639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:57.792611 25639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250: Bootstrap starting.
I20260806 06:54:57.793138 25639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:57.793375 25639 log.cc:824] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:57.794106 25639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250: No bootstrap required, opened a new log
I20260806 06:54:57.795847 25639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } }
I20260806 06:54:57.795995 25639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:57.796018 25639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704f39e825814c0da4ae5f681d972250, State: Initialized, Role: FOLLOWER
I20260806 06:54:57.796123 25639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } }
I20260806 06:54:57.796170 25639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:57.796190 25639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:57.796223 25639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:57.797044 25639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } }
I20260806 06:54:57.797137 25639 leader_election.cc:302] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 704f39e825814c0da4ae5f681d972250; no voters:
I20260806 06:54:57.797262 25639 leader_election.cc:288] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:57.797308 25642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:57.797483 25642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [term 1 LEADER]: Becoming Leader. State: Replica: 704f39e825814c0da4ae5f681d972250, State: Running, Role: LEADER
I20260806 06:54:57.797544 25639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:57.797642 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25571
I20260806 06:54:57.797670 25642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } }
I20260806 06:54:57.797807 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:57.798000 4062 external_mini_cluster.cc:1428] Setting key abaee954f6d4af64b35901c826d12943
I20260806 06:54:57.798070 25643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "704f39e825814c0da4ae5f681d972250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } } }
I20260806 06:54:57.798137 25643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:57.798291 25644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 704f39e825814c0da4ae5f681d972250. Latest consensus state: current_term: 1 leader_uuid: "704f39e825814c0da4ae5f681d972250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704f39e825814c0da4ae5f681d972250" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 39037 } } }
I20260806 06:54:57.798343 25647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:57.798346 25644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:57.798954 25647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:57.800693 25647 catalog_manager.cc:1347] Generated new cluster ID: 116d7867379e4bb1bf5c7bd66dfc8a8f
I20260806 06:54:57.800745 25647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:54:57.808508 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:57.798637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39183 (local address 127.3.247.190:39037)
0806 06:54:57.798818 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:57.798842 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:57.799109 (+ 267us) server_negotiation.cc:407] Connection header received
0806 06:54:57.799254 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:57.799263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:57.799331 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:57.799482 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:57.800489 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:57.801500 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:57.802361 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:57.802782 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:57.802917 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:57.805242 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:57.805246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:57.805246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:57.805264 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:57.806969 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:57.807349 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:57.807352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:57.807353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:57.807445 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:57.807669 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:57.807670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:57.807671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:57.807902 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:57.808097 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:57.808183 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:54:57.808290 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260806 06:54:57.809540 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:57.820142 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:57.810879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34531 (local address 127.3.247.190:39037)
0806 06:54:57.810923 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:57.810925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:57.813984 (+ 3059us) server_negotiation.cc:407] Connection header received
0806 06:54:57.814061 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:57.814062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:57.814085 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:57.814141 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:57.814737 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:57.815243 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:57.816004 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:57.816220 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:57.816289 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:57.817566 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:57.817567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:57.817568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:54:57.817574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:57.818967 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:57.819334 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:57.819336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:57.819336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:57.819390 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:54:57.819603 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:54:57.819604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:54:57.819605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:54:57.819715 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:57.819824 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:57.819953 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 06:54:57.820020 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:54:57.832175 25647 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:57.832806 25647 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:57.839541 25647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 704f39e825814c0da4ae5f681d972250: Generated new TSK 0
I20260806 06:54:57.839767 25647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:57.848344 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25571
W20260806 06:54:57.852723 25556 connection.cc:535] client connection to 127.3.247.190:39037 recv error: Network error: failed to read from TLS socket (remote: 127.3.247.190:39037): Connection reset by peer (error 104)
2026-08-06T06:54:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25670](info): set up 2 sockets
Aug 06 06:54:57 dist-test-slave-0kls krb5kdc[25670](info): commencing operation
krb5kdc: starting...
W20260806 06:54:58.878036 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:54:58 dist-test-slave-0kls krb5kdc[25670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:54:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:54:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:54:58.993203 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41557
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:41557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:54:59.073873 25683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:54:59.074097 25683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:54:59.074121 25683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:54:59.075230 25683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:54:59.075266 25683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:54:59.075279 25683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:54:59.075291 25683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:54:59.075304 25683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:54:59.076731 25683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:41557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:41557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:54:59.076925 25683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:54:59.077095 25683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:54:59.081965 25688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:54:59.081990 25691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:59.082070 25683 server_base.cc:1034] running on GCE node
W20260806 06:54:59.081990 25689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:54:59.082453 25683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:54:59.082764 25683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:54:59.083926 25683 hybrid_clock.cc:648] HybridClock initialized: now 1785999299083916 us; error 44 us; skew 500 ppm
Aug 06 06:54:59 dist-test-slave-0kls krb5kdc[25670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:54:59.090240 25683 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:54:59.090762 25683 webserver.cc:460] Webserver started at http://127.3.247.190:45265/ using document root <none> and password file <none>
I20260806 06:54:59.090993 25683 fs_manager.cc:359] Metadata directory not provided
I20260806 06:54:59.091058 25683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:54:59.091202 25683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:54:59.092347 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "38a8ef557948453da4232b23c5071462"
format_stamp: "Formatted at 2026-08-06 06:54:59 on dist-test-slave-0kls"
server_key: "8374d2caa0a23496e8d37b008e12b9a1"
server_key_iv: "00fc264a5149cc842de0a19ca5865076"
server_key_version: "encryptionkey@0"
I20260806 06:54:59.092708 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "38a8ef557948453da4232b23c5071462"
format_stamp: "Formatted at 2026-08-06 06:54:59 on dist-test-slave-0kls"
server_key: "8374d2caa0a23496e8d37b008e12b9a1"
server_key_iv: "00fc264a5149cc842de0a19ca5865076"
server_key_version: "encryptionkey@0"
I20260806 06:54:59.094172 25683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:54:59.094897 25698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:54:59.095072 25683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:54:59.095172 25683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "38a8ef557948453da4232b23c5071462"
format_stamp: "Formatted at 2026-08-06 06:54:59 on dist-test-slave-0kls"
server_key: "8374d2caa0a23496e8d37b008e12b9a1"
server_key_iv: "00fc264a5149cc842de0a19ca5865076"
server_key_version: "encryptionkey@0"
I20260806 06:54:59.095242 25683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:54:59.102072 25683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:54:59.103931 25683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:54:59.104079 25683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:54:59.108191 25683 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:41557
I20260806 06:54:59.108225 25750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:41557 every 8 connection(s)
I20260806 06:54:59.108588 25683 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:54:59.109442 25751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:54:59.111733 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462: Bootstrap starting.
I20260806 06:54:59.112303 25751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462: Neither blocks nor log segments found. Creating new log.
I20260806 06:54:59.112586 25751 log.cc:824] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462: Log is configured to *not* fsync() on all Append() calls
I20260806 06:54:59.113313 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462: No bootstrap required, opened a new log
I20260806 06:54:59.114979 25751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } }
I20260806 06:54:59.115118 25751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:54:59.115171 25751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a8ef557948453da4232b23c5071462, State: Initialized, Role: FOLLOWER
I20260806 06:54:59.115329 25751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } }
I20260806 06:54:59.115397 25751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:54:59.115447 25751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:54:59.115495 25751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:54:59.116328 25751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } }
I20260806 06:54:59.116442 25751 leader_election.cc:302] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a8ef557948453da4232b23c5071462; no voters:
I20260806 06:54:59.116607 25751 leader_election.cc:288] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:54:59.116637 25754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:54:59.116798 25754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [term 1 LEADER]: Becoming Leader. State: Replica: 38a8ef557948453da4232b23c5071462, State: Running, Role: LEADER
I20260806 06:54:59.116866 25751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:54:59.116936 25754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } }
I20260806 06:54:59.117347 25756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a8ef557948453da4232b23c5071462. Latest consensus state: current_term: 1 leader_uuid: "38a8ef557948453da4232b23c5071462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } } }
I20260806 06:54:59.117444 25756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:59.117668 25763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:54:59.117879 25755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a8ef557948453da4232b23c5071462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a8ef557948453da4232b23c5071462" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 41557 } } }
I20260806 06:54:59.118058 25755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462 [sys.catalog]: This master's current role is: LEADER
I20260806 06:54:59.118134 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25683
I20260806 06:54:59.118238 25763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:54:59.118254 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:54:59.118465 4062 external_mini_cluster.cc:1428] Setting key a95ef8e08a881ebcc2f9512aa438938b
I20260806 06:54:59.119916 25763 catalog_manager.cc:1347] Generated new cluster ID: dc3f8efdb4eb4618a73d26376f4ba12a
I20260806 06:54:59.119963 25763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:54:59.123483 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:59.119086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44891 (local address 127.3.247.190:41557)
0806 06:54:59.119229 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:59.119237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:59.119274 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:54:59.119401 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:59.119408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:59.119463 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:59.119623 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:59.119860 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:59.120772 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:59.121553 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:59.122087 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:59.122200 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:59.122896 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:59.122900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:59.122900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:59.123073 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:59.123102 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:59.123113 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:59.123200 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:59.123292 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:54:59.123320 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260806 06:54:59.124406 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:54:59.127046 25763 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:54:59.127735 25763 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:54:59.129558 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:54:59.126050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34249 (local address 127.3.247.190:41557)
0806 06:54:59.126086 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:54:59.126087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:54:59.126741 (+ 654us) server_negotiation.cc:407] Connection header received
0806 06:54:59.126896 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:54:59.126897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:54:59.126908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:54:59.126968 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:54:59.127136 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:59.127628 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:59.128334 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:54:59.128566 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:54:59.128639 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:54:59.129292 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:54:59.129293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:54:59.129293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:54:59.129298 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:54:59.129317 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:54:59.129321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:54:59.129387 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:54:59.129450 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:54:59.129462 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:54:59.131275 25763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38a8ef557948453da4232b23c5071462: Generated new TSK 0
I20260806 06:54:59.131435 25763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:54:59.136552 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25683
W20260806 06:54:59.141193 25668 connection.cc:535] client connection to 127.3.247.190:41557 recv error: Network error: failed to read from TLS socket (remote: 127.3.247.190:41557): Connection reset by peer (error 104)
2026-08-06T06:54:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:54:59 dist-test-slave-0kls krb5kdc[25782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:54:59 dist-test-slave-0kls krb5kdc[25782](info): set up 2 sockets
Aug 06 06:54:59 dist-test-slave-0kls krb5kdc[25782](info): commencing operation
krb5kdc: starting...
W20260806 06:55:00.165961 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:00 dist-test-slave-0kls krb5kdc[25782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:00.278419 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38909
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:00.358430 25795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:00.358623 25795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:00.358645 25795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:00.359730 25795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:00.359761 25795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:00.359776 25795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:00.359787 25795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:00.359799 25795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:00.361141 25795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:38909
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:38909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:00.361321 25795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:00.361507 25795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:55:00.366346 25795 server_base.cc:1034] running on GCE node
W20260806 06:55:00.366289 25801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:00.366289 25800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:00.366289 25803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:00.366753 25795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:00.367054 25795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:00.368219 25795 hybrid_clock.cc:648] HybridClock initialized: now 1785999300368196 us; error 42 us; skew 500 ppm
Aug 06 06:55:00 dist-test-slave-0kls krb5kdc[25782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999300, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:00.370820 25795 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:00.371269 25795 webserver.cc:460] Webserver started at http://127.3.247.190:43751/ using document root <none> and password file <none>
I20260806 06:55:00.371461 25795 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:00.371501 25795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:00.371590 25795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:00.372586 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "a72dd096927a4d77af7ef6c284df6813"
format_stamp: "Formatted at 2026-08-06 06:55:00 on dist-test-slave-0kls"
server_key: "4e626f284f238ac80c27817449ab32a5"
server_key_iv: "ab1cc8fd1aead69164515960ae2671c3"
server_key_version: "encryptionkey@0"
I20260806 06:55:00.372884 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "a72dd096927a4d77af7ef6c284df6813"
format_stamp: "Formatted at 2026-08-06 06:55:00 on dist-test-slave-0kls"
server_key: "4e626f284f238ac80c27817449ab32a5"
server_key_iv: "ab1cc8fd1aead69164515960ae2671c3"
server_key_version: "encryptionkey@0"
I20260806 06:55:00.374234 25795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:55:00.375000 25810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:00.375195 25795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:55:00.375283 25795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "a72dd096927a4d77af7ef6c284df6813"
format_stamp: "Formatted at 2026-08-06 06:55:00 on dist-test-slave-0kls"
server_key: "4e626f284f238ac80c27817449ab32a5"
server_key_iv: "ab1cc8fd1aead69164515960ae2671c3"
server_key_version: "encryptionkey@0"
I20260806 06:55:00.375339 25795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:00.378614 25795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:00.380789 25795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:00.380896 25795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:00.384622 25795 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:38909
I20260806 06:55:00.384676 25862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:38909 every 8 connection(s)
I20260806 06:55:00.384972 25795 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:00.385991 25863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:00.388156 25863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813: Bootstrap starting.
I20260806 06:55:00.388731 25863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:00.389005 25863 log.cc:824] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:00.389652 25863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813: No bootstrap required, opened a new log
I20260806 06:55:00.391245 25863 raft_consensus.cc:348] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } }
I20260806 06:55:00.391419 25863 raft_consensus.cc:374] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:55:00.391470 25863 raft_consensus.cc:729] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72dd096927a4d77af7ef6c284df6813, State: Initialized, Role: FOLLOWER
I20260806 06:55:00.391611 25863 consensus_queue.cc:260] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } }
I20260806 06:55:00.391687 25863 raft_consensus.cc:388] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:55:00.391729 25863 raft_consensus.cc:482] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:55:00.391778 25863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:00.392555 25863 raft_consensus.cc:504] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } }
I20260806 06:55:00.392668 25863 leader_election.cc:302] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a72dd096927a4d77af7ef6c284df6813; no voters:
I20260806 06:55:00.392729 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25795
I20260806 06:55:00.392845 25863 leader_election.cc:288] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:55:00.392858 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:00.392863 25866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:00.392976 25866 raft_consensus.cc:686] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [term 1 LEADER]: Becoming Leader. State: Replica: a72dd096927a4d77af7ef6c284df6813, State: Running, Role: LEADER
I20260806 06:55:00.393082 4062 external_mini_cluster.cc:1428] Setting key 644845026509a0e2260dab5e6381188f
I20260806 06:55:00.393105 25863 sys_catalog.cc:564] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:00.393134 25866 consensus_queue.cc:237] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } }
I20260806 06:55:00.393567 25868 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a72dd096927a4d77af7ef6c284df6813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } } }
I20260806 06:55:00.393666 25868 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:00.393550 25867 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a72dd096927a4d77af7ef6c284df6813. Latest consensus state: current_term: 1 leader_uuid: "a72dd096927a4d77af7ef6c284df6813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72dd096927a4d77af7ef6c284df6813" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 38909 } } }
I20260806 06:55:00.393998 25867 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:00.394233 25874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:00.394848 25874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:55:00.395566 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:00.393574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53181 (local address 127.3.247.190:38909)
0806 06:55:00.393936 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:00.393945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:00.393976 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:55:00.394104 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:00.394112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:00.394155 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:00.394604 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:00.395144 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:00.395149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:00.395149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:00.395152 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:00.395198 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:00.395213 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:00.395270 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:00.395352 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:55:00.395391 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":59,"threads_started":1}
I20260806 06:55:00.396417 25874 catalog_manager.cc:1347] Generated new cluster ID: e7ecda8e3de847f7aede1fcb96fb915a
I20260806 06:55:00.396463 25874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:55:00.396466 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:55:00.402931 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:00.401901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35847 (local address 127.3.247.190:38909)
0806 06:55:00.401938 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:00.401940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:00.401949 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:55:00.401971 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:00.401971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:00.401976 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:00.402036 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:00.402678 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:00.402679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:00.402680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:00.402684 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:00.402700 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:00.402704 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:00.402775 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:00.402821 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 06:55:00.402833 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:55:00.408720 25874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:00.408890 25874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:00.419054 25874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a72dd096927a4d77af7ef6c284df6813: Generated new TSK 0
I20260806 06:55:00.419241 25874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:55:00.426028 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25795
2026-08-06T06:55:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:55:00 dist-test-slave-0kls krb5kdc[25894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:55:00 dist-test-slave-0kls krb5kdc[25894](info): set up 2 sockets
Aug 06 06:55:00 dist-test-slave-0kls krb5kdc[25894](info): commencing operation
krb5kdc: starting...
W20260806 06:55:01.446108 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[25894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:01.560309 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40611
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:01.639953 25907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:01.640201 25907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:01.640240 25907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:01.641410 25907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:01.641481 25907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:01.641505 25907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:01.641527 25907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:01.641567 25907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:01.643079 25907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:01.643352 25907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:01.643579 25907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:01.648398 25913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:01.648391 25912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:01.648391 25915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:01.648624 25907 server_base.cc:1034] running on GCE node
I20260806 06:55:01.648854 25907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:01.649156 25907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:01.650336 25907 hybrid_clock.cc:648] HybridClock initialized: now 1785999301650316 us; error 36 us; skew 500 ppm
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[25894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999301, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:01.652803 25907 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:01.653246 25907 webserver.cc:460] Webserver started at http://127.3.247.190:44301/ using document root <none> and password file <none>
I20260806 06:55:01.653452 25907 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:01.653494 25907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:01.653582 25907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:01.654656 25907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "5237c0e4169140b18ba01951df96ffbc"
format_stamp: "Formatted at 2026-08-06 06:55:01 on dist-test-slave-0kls"
server_key: "0f06f01690f3052c4aba55b8a8789f7d"
server_key_iv: "b7162889f72dd25d2471116965d94c69"
server_key_version: "encryptionkey@0"
I20260806 06:55:01.654974 25907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "5237c0e4169140b18ba01951df96ffbc"
format_stamp: "Formatted at 2026-08-06 06:55:01 on dist-test-slave-0kls"
server_key: "0f06f01690f3052c4aba55b8a8789f7d"
server_key_iv: "b7162889f72dd25d2471116965d94c69"
server_key_version: "encryptionkey@0"
I20260806 06:55:01.656216 25907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:55:01.657002 25922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:01.657188 25907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:55:01.657267 25907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "5237c0e4169140b18ba01951df96ffbc"
format_stamp: "Formatted at 2026-08-06 06:55:01 on dist-test-slave-0kls"
server_key: "0f06f01690f3052c4aba55b8a8789f7d"
server_key_iv: "b7162889f72dd25d2471116965d94c69"
server_key_version: "encryptionkey@0"
I20260806 06:55:01.657351 25907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:01.674681 25907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:01.676891 25907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:01.677006 25907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:01.680999 25907 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40611
I20260806 06:55:01.681057 25974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40611 every 8 connection(s)
I20260806 06:55:01.681403 25907 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:01.682308 25975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:01.684556 25975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc: Bootstrap starting.
I20260806 06:55:01.685160 25975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:01.685293 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 25907
I20260806 06:55:01.685432 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:01.685469 25975 log.cc:824] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:01.685654 4062 external_mini_cluster.cc:1428] Setting key 252cda3cbad92f0660907f928252b557
I20260806 06:55:01.686371 25975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc: No bootstrap required, opened a new log
I20260806 06:55:01.688920 25975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } }
I20260806 06:55:01.689087 25975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:55:01.689122 25975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5237c0e4169140b18ba01951df96ffbc, State: Initialized, Role: FOLLOWER
I20260806 06:55:01.689242 25975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } }
I20260806 06:55:01.689307 25975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:55:01.689334 25975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:55:01.689375 25975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:01.690044 25975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } }
I20260806 06:55:01.690148 25975 leader_election.cc:302] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5237c0e4169140b18ba01951df96ffbc; no voters:
I20260806 06:55:01.690302 25975 leader_election.cc:288] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:55:01.690330 25980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:01.690516 25980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [term 1 LEADER]: Becoming Leader. State: Replica: 5237c0e4169140b18ba01951df96ffbc, State: Running, Role: LEADER
I20260806 06:55:01.690613 25975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:01.690641 25980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } }
I20260806 06:55:01.690969 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5237c0e4169140b18ba01951df96ffbc. Latest consensus state: current_term: 1 leader_uuid: "5237c0e4169140b18ba01951df96ffbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } } }
I20260806 06:55:01.691045 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:01.690996 25981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5237c0e4169140b18ba01951df96ffbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5237c0e4169140b18ba01951df96ffbc" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40611 } } }
I20260806 06:55:01.691217 25981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:01.691268 25986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:01.692102 25986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[25894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:01.694001 25986 catalog_manager.cc:1347] Generated new cluster ID: 852c093da9d148b084569c06264bcc61
I20260806 06:55:01.694051 25986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:55:01.698429 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:01.686186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57817 (local address 127.3.247.190:40611)
0806 06:55:01.686349 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:01.686358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:01.686392 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:55:01.686833 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:01.686840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:01.686886 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:01.686998 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:01.687778 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:01.688637 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:01.689679 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:01.690185 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:01.690310 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:01.694437 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:01.694441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:01.694442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:01.694455 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:01.696391 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:01.696783 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:01.696786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:01.696786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:01.696884 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:01.697092 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:01.697093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:01.697093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:01.697398 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:01.697542 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:01.698060 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 06:55:01.698200 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260806 06:55:01.699481 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:55:01.708237 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:01.700990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47099 (local address 127.3.247.190:40611)
0806 06:55:01.701026 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:01.701027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:01.701116 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:55:01.701207 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:01.701208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:01.701217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:01.701245 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:01.701965 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:01.702707 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:01.703731 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:01.704061 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:01.704103 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:01.705528 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:01.705529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:01.705529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:01.705536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:01.707052 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:01.707436 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:01.707437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:01.707437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:01.707486 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:01.707671 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:01.707671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:01.707672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:01.707789 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:01.707857 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:01.708065 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:55:01.708139 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:55:01.714327 25986 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:01.714871 25986 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:01.720640 25986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5237c0e4169140b18ba01951df96ffbc: Generated new TSK 0
I20260806 06:55:01.720829 25986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:55:01.723945 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 25907
2026-08-06T06:55:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[26006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[26006](info): set up 2 sockets
Aug 06 06:55:01 dist-test-slave-0kls krb5kdc[26006](info): commencing operation
krb5kdc: starting...
W20260806 06:55:02.758002 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:02 dist-test-slave-0kls krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:02.873040 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40463
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:40463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:02.954442 26019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:02.954640 26019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:02.954663 26019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:02.955760 26019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:02.955794 26019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:02.955809 26019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:02.955822 26019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:02.955833 26019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:02.957180 26019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:40463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:40463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:02.957439 26019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:02.957676 26019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:02.962363 26025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:02.962356 26024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:02.962415 26027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:02.962536 26019 server_base.cc:1034] running on GCE node
I20260806 06:55:02.962807 26019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:02.963104 26019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:02.964250 26019 hybrid_clock.cc:648] HybridClock initialized: now 1785999302964242 us; error 32 us; skew 500 ppm
Aug 06 06:55:02 dist-test-slave-0kls krb5kdc[26006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:02.966580 26019 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:02.967020 26019 webserver.cc:460] Webserver started at http://127.3.247.190:34701/ using document root <none> and password file <none>
I20260806 06:55:02.967279 26019 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:02.967358 26019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:02.967489 26019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:02.968560 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "ee9bb37bcaf141e4acb16f5ae1dad039"
format_stamp: "Formatted at 2026-08-06 06:55:02 on dist-test-slave-0kls"
server_key: "f5e932bd254bc2f0af2291eb57d146c9"
server_key_iv: "f7e8bbfceec02974e697e72aca1f5d2d"
server_key_version: "encryptionkey@0"
I20260806 06:55:02.968904 26019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "ee9bb37bcaf141e4acb16f5ae1dad039"
format_stamp: "Formatted at 2026-08-06 06:55:02 on dist-test-slave-0kls"
server_key: "f5e932bd254bc2f0af2291eb57d146c9"
server_key_iv: "f7e8bbfceec02974e697e72aca1f5d2d"
server_key_version: "encryptionkey@0"
I20260806 06:55:02.970242 26019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:55:02.971052 26034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:02.971256 26019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:55:02.971367 26019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "ee9bb37bcaf141e4acb16f5ae1dad039"
format_stamp: "Formatted at 2026-08-06 06:55:02 on dist-test-slave-0kls"
server_key: "f5e932bd254bc2f0af2291eb57d146c9"
server_key_iv: "f7e8bbfceec02974e697e72aca1f5d2d"
server_key_version: "encryptionkey@0"
I20260806 06:55:02.971441 26019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:02.991556 26019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:02.993337 26019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:02.993484 26019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:02.997525 26019 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:40463
I20260806 06:55:02.997578 26086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:40463 every 8 connection(s)
I20260806 06:55:02.997963 26019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:02.998760 26087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:03.000983 26087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039: Bootstrap starting.
I20260806 06:55:03.001569 26087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:03.001911 26087 log.cc:824] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:03.002612 26087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039: No bootstrap required, opened a new log
I20260806 06:55:03.004179 26087 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } }
I20260806 06:55:03.004316 26087 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:55:03.004341 26087 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bb37bcaf141e4acb16f5ae1dad039, State: Initialized, Role: FOLLOWER
I20260806 06:55:03.004447 26087 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } }
I20260806 06:55:03.004498 26087 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:55:03.004518 26087 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:55:03.004547 26087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:03.005347 26087 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } }
I20260806 06:55:03.005435 26087 leader_election.cc:302] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee9bb37bcaf141e4acb16f5ae1dad039; no voters:
I20260806 06:55:03.005559 26087 leader_election.cc:288] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:55:03.005590 26090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:03.005735 26090 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [term 1 LEADER]: Becoming Leader. State: Replica: ee9bb37bcaf141e4acb16f5ae1dad039, State: Running, Role: LEADER
I20260806 06:55:03.005865 26087 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:03.005894 26090 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } }
I20260806 06:55:03.006379 26092 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee9bb37bcaf141e4acb16f5ae1dad039. Latest consensus state: current_term: 1 leader_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } } }
I20260806 06:55:03.006484 26092 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:03.006719 26091 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9bb37bcaf141e4acb16f5ae1dad039" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 40463 } } }
I20260806 06:55:03.006821 26091 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:03.006958 26099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:03.007474 26099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:55:03.008049 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26019
I20260806 06:55:03.008172 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:03.008345 4062 external_mini_cluster.cc:1428] Setting key dfc318970f61e8da8508bbc17dfb6ce3
I20260806 06:55:03.009135 26099 catalog_manager.cc:1347] Generated new cluster ID: ca20a17f54924acd947d90d87a9c2927
I20260806 06:55:03.009191 26099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:55:03.014561 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:03.008844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60889 (local address 127.3.247.190:40463)
0806 06:55:03.008995 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:03.009005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:03.009091 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:55:03.009243 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:03.009250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:03.009306 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:03.009456 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:03.009677 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:03.011107 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:03.012175 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:03.012801 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:03.012914 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:03.013640 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:03.013644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:03.013644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:03.013892 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:03.013932 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:03.013950 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:03.014032 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:03.014096 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:55:03.014162 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260806 06:55:03.015545 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:55:03.016197 26099 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:03.016865 26099 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:03.020409 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:03.016999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35215 (local address 127.3.247.190:40463)
0806 06:55:03.017037 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:03.017039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:03.017157 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:55:03.017187 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:03.017188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:03.017198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:03.017263 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:03.017396 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:03.018142 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:03.019135 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:03.019447 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:03.019527 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:03.020170 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:03.020172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:03.020172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:03.020176 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:03.020191 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:03.020195 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:03.020245 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:03.020303 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:55:03.020327 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:55:03.026070 26099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee9bb37bcaf141e4acb16f5ae1dad039: Generated new TSK 0
I20260806 06:55:03.026260 26099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:55:03.033170 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26019
2026-08-06T06:55:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:55:03 dist-test-slave-0kls krb5kdc[26118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:55:03 dist-test-slave-0kls krb5kdc[26118](info): set up 2 sockets
Aug 06 06:55:03 dist-test-slave-0kls krb5kdc[26118](info): commencing operation
krb5kdc: starting...
W20260806 06:55:04.070079 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:04 dist-test-slave-0kls krb5kdc[26118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:04.186662 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34547
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:42289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:34547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:04.266316 26131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:04.266541 26131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:04.266562 26131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:04.267637 26131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:04.267673 26131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:04.267686 26131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:04.267699 26131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:04.267710 26131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:04.269068 26131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:42289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:34547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:34547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:04.269253 26131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:04.269420 26131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:04.274204 26139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:04.274223 26137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:04.274231 26136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:04.274449 26131 server_base.cc:1034] running on GCE node
I20260806 06:55:04.274667 26131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:04.274941 26131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:04.276075 26131 hybrid_clock.cc:648] HybridClock initialized: now 1785999304276062 us; error 29 us; skew 500 ppm
Aug 06 06:55:04 dist-test-slave-0kls krb5kdc[26118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:04.283418 26131 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:04.283895 26131 webserver.cc:460] Webserver started at http://127.3.247.190:40775/ using document root <none> and password file <none>
I20260806 06:55:04.284130 26131 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:04.284176 26131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:04.284336 26131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:04.285398 26131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "b54592189b25480d8788c274ad71c884"
format_stamp: "Formatted at 2026-08-06 06:55:04 on dist-test-slave-0kls"
server_key: "ddcea3a75c15f62a18a9c9aa9b9e6915"
server_key_iv: "0dcc83465225db06e5e8f9ec14481405"
server_key_version: "encryptionkey@0"
I20260806 06:55:04.285744 26131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "b54592189b25480d8788c274ad71c884"
format_stamp: "Formatted at 2026-08-06 06:55:04 on dist-test-slave-0kls"
server_key: "ddcea3a75c15f62a18a9c9aa9b9e6915"
server_key_iv: "0dcc83465225db06e5e8f9ec14481405"
server_key_version: "encryptionkey@0"
I20260806 06:55:04.287051 26131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:55:04.287721 26146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:04.287904 26131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:55:04.287982 26131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "b54592189b25480d8788c274ad71c884"
format_stamp: "Formatted at 2026-08-06 06:55:04 on dist-test-slave-0kls"
server_key: "ddcea3a75c15f62a18a9c9aa9b9e6915"
server_key_iv: "0dcc83465225db06e5e8f9ec14481405"
server_key_version: "encryptionkey@0"
I20260806 06:55:04.288051 26131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:04.291417 26131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:04.293148 26131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:04.293306 26131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:04.297231 26131 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:34547
I20260806 06:55:04.297260 26198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:34547 every 8 connection(s)
I20260806 06:55:04.297587 26131 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:04.298537 26199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:04.300678 26199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884: Bootstrap starting.
I20260806 06:55:04.300755 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26131
I20260806 06:55:04.300916 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:04.301102 4062 external_mini_cluster.cc:1428] Setting key f7e4898d763fdc003283e380b1b4433f
I20260806 06:55:04.301345 26199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:04.301652 26199 log.cc:824] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:04.302259 26199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884: No bootstrap required, opened a new log
I20260806 06:55:04.303390 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:04.301558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47277 (local address 127.3.247.190:34547)
0806 06:55:04.301741 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:04.301750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:04.301799 (+ 49us) server_negotiation.cc:407] Connection header received
0806 06:55:04.302206 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:04.302212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:04.302235 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:04.302406 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:04.302982 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:04.302986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:04.302986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:04.302990 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:04.303037 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:04.303047 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:04.303143 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:04.303180 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:55:04.303218 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260806 06:55:04.304275 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:55:04.304580 26199 raft_consensus.cc:348] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } }
I20260806 06:55:04.304729 26199 raft_consensus.cc:374] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:55:04.304785 26199 raft_consensus.cc:729] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54592189b25480d8788c274ad71c884, State: Initialized, Role: FOLLOWER
I20260806 06:55:04.304898 26199 consensus_queue.cc:260] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } }
I20260806 06:55:04.305011 26199 raft_consensus.cc:388] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:55:04.305053 26199 raft_consensus.cc:482] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:55:04.305102 26199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:04.305855 26199 raft_consensus.cc:504] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } }
I20260806 06:55:04.305961 26199 leader_election.cc:302] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b54592189b25480d8788c274ad71c884; no voters:
I20260806 06:55:04.306120 26199 leader_election.cc:288] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:55:04.306149 26209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:04.306344 26209 raft_consensus.cc:686] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [term 1 LEADER]: Becoming Leader. State: Replica: b54592189b25480d8788c274ad71c884, State: Running, Role: LEADER
I20260806 06:55:04.306358 26199 sys_catalog.cc:564] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:04.306486 26209 consensus_queue.cc:237] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } }
I20260806 06:55:04.306893 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:04.305802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59779 (local address 127.3.247.190:34547)
0806 06:55:04.305842 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:04.305844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:04.305938 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:55:04.306001 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:04.306002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:04.306007 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:04.306061 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:04.306679 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:04.306681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:04.306681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:55:04.306685 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:04.306699 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:55:04.306704 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:04.306751 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:04.306817 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:55:04.306829 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:55:04.306929 26211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b54592189b25480d8788c274ad71c884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } } }
I20260806 06:55:04.307040 26211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:04.306890 26209 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b54592189b25480d8788c274ad71c884. Latest consensus state: current_term: 1 leader_uuid: "b54592189b25480d8788c274ad71c884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54592189b25480d8788c274ad71c884" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 34547 } } }
I20260806 06:55:04.307159 26209 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:04.307482 26216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:04.308007 26216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:55:04.309494 26216 catalog_manager.cc:1347] Generated new cluster ID: 434e8999a2654dc7a628c89c15ed184a
I20260806 06:55:04.309541 26216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:55:04.320914 26216 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:04.321089 26216 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:04.332998 26216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b54592189b25480d8788c274ad71c884: Generated new TSK 0
I20260806 06:55:04.333250 26216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:55:04.337877 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26131
2026-08-06T06:55:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1304 ms)
[----------] 18 tests from AuthTokenIssuingTest (23533 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:55:04 dist-test-slave-0kls krb5kdc[26230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:55:04 dist-test-slave-0kls krb5kdc[26230](info): set up 2 sockets
Aug 06 06:55:04 dist-test-slave-0kls krb5kdc[26230](info): commencing operation
krb5kdc: starting...
W20260806 06:55:05.374110 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:05.489892 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36369
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:05.570276 26243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:05.570554 26243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:05.570583 26243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:05.570627 26243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:05.571789 26243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:05.571856 26243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:05.571878 26243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:05.571899 26243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:05.571931 26243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:05.573526 26243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:05.573881 26243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:05.574148 26243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:05.579073 26249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:05.579110 26251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:05.579128 26248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:05.579355 26243 server_base.cc:1034] running on GCE node
I20260806 06:55:05.579620 26243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:05.579947 26243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:05.581128 26243 hybrid_clock.cc:648] HybridClock initialized: now 1785999305581101 us; error 36 us; skew 500 ppm
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:05.583658 26243 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:05.584172 26243 webserver.cc:460] Webserver started at http://127.3.247.190:32783/ using document root <none> and password file <none>
I20260806 06:55:05.584419 26243 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:05.584487 26243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:05.584632 26243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:05.585666 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "31e7b89ab4b14bc1917a9fa63cf2175c"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "46aac0c8153e612c0758ad36d7c41e60"
server_key_iv: "5a1602a276667c1d28e8f3637527220c"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.586081 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "31e7b89ab4b14bc1917a9fa63cf2175c"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "46aac0c8153e612c0758ad36d7c41e60"
server_key_iv: "5a1602a276667c1d28e8f3637527220c"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.587421 26243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:55:05.588157 26258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:05.588346 26243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:55:05.588446 26243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "31e7b89ab4b14bc1917a9fa63cf2175c"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "46aac0c8153e612c0758ad36d7c41e60"
server_key_iv: "5a1602a276667c1d28e8f3637527220c"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.588529 26243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:05.598393 26243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:05.600658 26243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:05.600822 26243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:05.604799 26243 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:36369
I20260806 06:55:05.604832 26310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:36369 every 8 connection(s)
I20260806 06:55:05.605163 26243 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:05.606146 26311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:05.607615 26311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:05.614614 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26243
I20260806 06:55:05.614729 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:05.614971 4062 external_mini_cluster.cc:1428] Setting key 6c80eae23f144b062d72871cfdee344a
I20260806 06:55:05.617516 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.608276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.3.247.190:36369)
0806 06:55:05.608479 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.608489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.608610 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:55:05.609150 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.609152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.609174 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.609270 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.609876 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.610651 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.611518 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.611828 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.611946 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.613981 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.613988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.613988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.613994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.615930 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.616316 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.616319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.616320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.616442 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.616716 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.616716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.616717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.616922 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.616985 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.617216 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.617334 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260806 06:55:05.617616 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.608249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36369 (local address 127.0.0.1:44840)
0806 06:55:05.608469 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.608512 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.608997 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.609275 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.609277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.609565 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.609708 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.609712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.610760 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.610761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.611423 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.611424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.611938 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.611939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.612001 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.612513 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.612521 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.613903 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.616030 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.616037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.616039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.616235 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.616522 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.616523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.616523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.616643 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.616988 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.616995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.617077 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.617183 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.617427 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":63,"thread_start_us":84,"threads_started":1}
I20260806 06:55:05.618296 26311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:05.618681 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.618547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33078)
0806 06:55:05.618597 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.618649 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:55:05.618844 26262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
I20260806 06:55:05.619284 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.619176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33090)
0806 06:55:05.619228 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.619257 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:55:05.619418 26311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41535: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.247.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
I20260806 06:55:05.638914 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:41535
--webserver_interface=127.3.247.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:55:05.668639 26311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } attempt: 1
I20260806 06:55:05.669104 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.668976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33106)
0806 06:55:05.669036 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.669074 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:55:05.669433 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.669338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33108)
0806 06:55:05.669379 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.669416 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:55:05.669518 26311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41535: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
W20260806 06:55:05.719223 26318 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:05.719482 26318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:05.719514 26318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:05.719555 26318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:05.720697 26318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:05.720769 26318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:05.720794 26318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:05.720824 26318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:05.720847 26318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:05.722348 26318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:41535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.247.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:05.722623 26318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:05.722853 26318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:05.727679 26324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:05.727689 26323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:05.727692 26326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:05.727958 26318 server_base.cc:1034] running on GCE node
I20260806 06:55:05.728202 26318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:05.728483 26318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:05.729657 26318 hybrid_clock.cc:648] HybridClock initialized: now 1785999305729644 us; error 28 us; skew 500 ppm
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:05.732487 26318 init.cc:377] Logged in from keytab as kudu/127.3.247.189@KRBTEST.COM (short username kudu)
I20260806 06:55:05.732987 26318 webserver.cc:460] Webserver started at http://127.3.247.189:32851/ using document root <none> and password file <none>
I20260806 06:55:05.733215 26318 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:05.733282 26318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:05.733454 26318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:05.734609 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data/instance:
uuid: "b5373a15e1aa40e48e6e7a666b7ed39a"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "b18d3270934ca78c32ff85bcbe52c187"
server_key_iv: "cd8801f106e3c18c051cc4176c638223"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.734961 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal/instance:
uuid: "b5373a15e1aa40e48e6e7a666b7ed39a"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "b18d3270934ca78c32ff85bcbe52c187"
server_key_iv: "cd8801f106e3c18c051cc4176c638223"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.736248 26318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:55:05.736956 26333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:05.737120 26318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:55:05.737221 26318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "b5373a15e1aa40e48e6e7a666b7ed39a"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "b18d3270934ca78c32ff85bcbe52c187"
server_key_iv: "cd8801f106e3c18c051cc4176c638223"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.737293 26318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:05.737643 26311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } attempt: 2
I20260806 06:55:05.738019 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.737900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33112)
0806 06:55:05.737952 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.737983 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:55:05.738315 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.738234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33118)
0806 06:55:05.738273 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.738296 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:55:05.738428 26311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41535: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41535: connect: Connection refused (error 111)
I20260806 06:55:05.759507 26318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:05.761907 26318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:05.762059 26318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:05.765954 26318 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.189:41535
I20260806 06:55:05.766021 26385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.189:41535 every 8 connection(s)
I20260806 06:55:05.766376 26318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
I20260806 06:55:05.767373 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:05.768909 26386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:05.773438 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26318
I20260806 06:55:05.773576 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-1/wal/instance
I20260806 06:55:05.773819 4062 external_mini_cluster.cc:1428] Setting key 9ba7185ab9668da618d5af969478ebad
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:05.778282 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.769611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.3.247.190:36369)
0806 06:55:05.769669 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.769671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.769895 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:55:05.770398 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.770400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.770412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.770482 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.771039 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.771532 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.772480 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.772662 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.772711 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.775389 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.775390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.775391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.775399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.776748 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.777241 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.777242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.777243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.777300 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.777583 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.777584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.777584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.777694 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.777748 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.778140 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.778213 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:55:05.778543 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.769585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36369 (local address 127.0.0.1:44850)
0806 06:55:05.769734 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.769776 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.770272 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.770536 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.770540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.770774 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.770964 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.770968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.771610 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.771610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.772392 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.772393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.772740 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.772741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.772842 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.773377 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.773385 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.775262 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.776828 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.776834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.776837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.777166 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.777365 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.777366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.777366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.777515 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.777763 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.777770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.777990 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.778119 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.778301 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260806 06:55:05.779173 26386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.247.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.188@KRBTEST.COM" created.
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:05.788092 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.779479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33120)
0806 06:55:05.779536 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.779545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.779590 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.779827 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.779828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.779969 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.780012 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.780013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.780846 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.780847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.781349 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.781350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.781820 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.781821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.781883 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.782305 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.782309 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.784982 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.787082 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.787083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.787083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.787240 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.787450 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.787450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.787450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.787498 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.787806 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.787807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.787840 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.787938 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.788031 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:55:05.788259 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.779546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.3.247.189:41535)
0806 06:55:05.779693 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.779696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.779711 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:55:05.779726 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.779728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.779738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.779813 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.780091 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.780745 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.781434 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.781725 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.781816 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.785073 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.785079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.785079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.785085 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.787002 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.787318 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.787321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.787321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.787384 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.787556 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.787557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.787558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.787741 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.787827 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.787987 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.788117 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260806 06:55:05.788975 26386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:05.789343 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.789209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45564)
0806 06:55:05.789262 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.789313 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:55:05.789491 26336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
I20260806 06:55:05.789934 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.789829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45578)
0806 06:55:05.789865 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.789896 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 06:55:05.790081 26386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43409: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
I20260806 06:55:05.798599 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:43409
--webserver_interface=127.3.247.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:55:05.829650 26311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } attempt: 3
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:05.836810 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.830064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33128)
0806 06:55:05.830127 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.830140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.830204 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.830365 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.830366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.830548 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.830614 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.830615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.831283 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.831284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.831812 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.831853 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.832411 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.832412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.832470 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.832949 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.832954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.834148 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.835916 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.835917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.835918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.836080 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.836275 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.836276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.836276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.836327 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.836578 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.836578 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.836616 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.836667 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.836739 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:55:05.836872 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.830162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.3.247.189:41535)
0806 06:55:05.830233 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.830236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.830245 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:55:05.830283 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.830284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.830295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.830359 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.830698 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.831196 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.831958 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.832308 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.832398 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.834241 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.834242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.834242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.834251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.835834 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.836150 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.836151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.836151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.836212 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.836384 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.836384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.836385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.836516 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.836581 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.836709 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.836781 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:55:05.837284 26311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:05.837658 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.837563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45588)
0806 06:55:05.837610 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.837640 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:55:05.837961 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.837900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45594)
0806 06:55:05.837929 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.837949 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:55:05.838057 26311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43409: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
I20260806 06:55:05.839293 26386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } attempt: 1
I20260806 06:55:05.839660 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.839551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45598)
0806 06:55:05.839603 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.839635 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:55:05.840050 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.839961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45614)
0806 06:55:05.840007 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.840032 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:55:05.840142 26386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43409: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
I20260806 06:55:05.869212 26311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } attempt: 1
I20260806 06:55:05.869721 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.869590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45616)
0806 06:55:05.869655 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.869694 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 06:55:05.870100 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.870017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45628)
0806 06:55:05.870059 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.870083 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:55:05.870210 26311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43409: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
W20260806 06:55:05.881318 26393 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:05.881536 26393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:05.881552 26393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:05.881572 26393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:05.882799 26393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:05.882838 26393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:05.882853 26393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:05.882897 26393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:05.882916 26393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:05.884750 26393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:36369,127.3.247.189:41535,127.3.247.188:43409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:43409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.247.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:05.885030 26393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:05.885268 26393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:05.890098 26401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:05.890092 26399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:05.890272 26393 server_base.cc:1034] running on GCE node
W20260806 06:55:05.890115 26398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:05.890606 26393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:05.890908 26393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:05.892091 26393 hybrid_clock.cc:648] HybridClock initialized: now 1785999305892079 us; error 47 us; skew 500 ppm
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:05.894637 26393 init.cc:377] Logged in from keytab as kudu/127.3.247.188@KRBTEST.COM (short username kudu)
I20260806 06:55:05.895023 26393 webserver.cc:460] Webserver started at http://127.3.247.188:41551/ using document root <none> and password file <none>
I20260806 06:55:05.895211 26393 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:05.895248 26393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:05.895331 26393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:05.896291 26393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data/instance:
uuid: "1e929d679eb64a60ac8481eed3366c2e"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "898be59ae5c3b7f5afc4cad2aacf9fc7"
server_key_iv: "91ee7d548d99891f1e6c200871b91b95"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.896582 26393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal/instance:
uuid: "1e929d679eb64a60ac8481eed3366c2e"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "898be59ae5c3b7f5afc4cad2aacf9fc7"
server_key_iv: "91ee7d548d99891f1e6c200871b91b95"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.897818 26393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:55:05.898530 26408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:05.898693 26393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:55:05.898797 26393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "1e929d679eb64a60ac8481eed3366c2e"
format_stamp: "Formatted at 2026-08-06 06:55:05 on dist-test-slave-0kls"
server_key: "898be59ae5c3b7f5afc4cad2aacf9fc7"
server_key_iv: "91ee7d548d99891f1e6c200871b91b95"
server_key_version: "encryptionkey@0"
I20260806 06:55:05.898873 26393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:05.906682 26393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:05.908267 26386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } attempt: 2
I20260806 06:55:05.908676 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.908550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45640)
0806 06:55:05.908610 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.908647 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:55:05.908974 26393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:05.909044 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.908941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45642)
0806 06:55:05.908987 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.909012 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:55:05.909142 26386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:43409: Network error: Client connection negotiation failed: client connection to 127.3.247.188:43409: connect: Connection refused (error 111)
I20260806 06:55:05.909159 26393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:05.913058 26393 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.188:43409
I20260806 06:55:05.913112 26460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.188:43409 every 8 connection(s)
I20260806 06:55:05.913475 26393 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
I20260806 06:55:05.914513 26461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:05.916054 26461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:05.923245 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26393
I20260806 06:55:05.923353 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785999040366271-4062-0/minicluster-data/master-2/wal/instance
I20260806 06:55:05.923578 4062 external_mini_cluster.cc:1428] Setting key a3a1cfb0cfe99ddf85eee0f880e5b5ed
I20260806 06:55:05.924726 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.916744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.3.247.190:36369)
0806 06:55:05.916827 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.916829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.916989 (+ 160us) server_negotiation.cc:407] Connection header received
0806 06:55:05.917444 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.917445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.917455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.917503 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.918120 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.918602 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.919528 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.919701 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.919750 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.921776 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.921777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.921777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.921819 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.923013 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.923519 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.923519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.923520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.923571 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.923907 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.923909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.923909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.924085 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.924153 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.924517 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.924616 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:55:05.924829 26463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.916700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:36369 (local address 127.0.0.1:44856)
0806 06:55:05.916865 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.916904 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.917343 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.917559 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.917562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.917857 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.918050 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.918054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.918678 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.918679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.919460 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.919461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.919779 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.919780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.919885 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.920324 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.920340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.921689 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.923090 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.923099 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.923100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.923416 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.923640 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.923641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.923642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.923800 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.924156 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.924164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.924332 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.924465 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.924642 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260806 06:55:05.925441 26461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:05.932523 26463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.925697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41535 (local address 127.0.0.1:33136)
0806 06:55:05.925748 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.925759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.925833 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.925958 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.925958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.926093 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.926141 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.926142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.926893 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.926894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.927521 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.927523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.927934 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.927935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.928008 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.928427 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.928432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.929572 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.931455 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.931456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.931457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.931702 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.931913 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.931914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.931914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.931968 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.932214 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.932215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.932256 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.932340 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.932429 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:55:05.932523 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.925757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.3.247.189:41535)
0806 06:55:05.925816 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.925818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.925827 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:55:05.925902 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.925903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.925910 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.925953 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.926196 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.926818 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.927598 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.927860 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.927932 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.929658 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.929659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.929660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.929667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.931339 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.931785 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.931786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.931787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.931838 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.932038 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.932039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.932040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.932149 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.932222 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.932366 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.932441 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:55:05.932886 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.924191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45656 (local address 127.3.247.188:43409)
0806 06:55:05.924365 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.924370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.924388 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:55:05.924409 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.924411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.924426 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.924499 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.925323 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.926309 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.927223 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.927669 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.927716 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.930209 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.930216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.930217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.930223 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.931734 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.932085 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.932087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.932088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.932140 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.932332 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.932332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.932333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.932469 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.932621 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.932688 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.932760 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260806 06:55:05.933462 26461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:05.940584 26463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.933714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45666)
0806 06:55:05.933832 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.933841 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.933889 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.934371 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.934371 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.934509 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.934537 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.934539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.935188 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.935189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.935698 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.935700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.936102 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.936103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.936173 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.936567 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.936572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.937751 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.939562 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.939562 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.939563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.939724 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.939941 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.939942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.939942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.939992 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.940259 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.940260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.940295 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.940386 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.940482 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 06:55:05.940632 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.933755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.3.247.188:43409)
0806 06:55:05.933813 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.933814 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.934284 (+ 470us) server_negotiation.cc:407] Connection header received
0806 06:55:05.934301 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.934302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.934309 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.934347 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.934608 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.935114 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.935770 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.936028 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.936099 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.937880 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.937881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.937881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.937888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.939485 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.939810 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.939811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.939811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.939859 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.940082 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.940083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.940083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.940183 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.940255 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.940425 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.940525 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":17}
I20260806 06:55:05.941453 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.933688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.3.247.190:36369)
0806 06:55:05.933726 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.933728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.933736 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:55:05.933821 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.933822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.933832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.933888 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.934284 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.934887 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.935619 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.935892 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.935940 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.938532 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.938533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.938533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.938540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.939956 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.940301 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.940302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.940302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.940379 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.940582 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.940583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.940583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.940745 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.940915 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.941197 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.941280 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:55:05.942588 26461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: Bootstrap starting.
I20260806 06:55:05.943495 26461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:05.943821 26461 log.cc:824] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:05.944618 26461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: No bootstrap required, opened a new log
I20260806 06:55:05.945425 26311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } attempt: 2
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:05.946976 26461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.947126 26461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e929d679eb64a60ac8481eed3366c2e, State: Initialized, Role: FOLLOWER
I20260806 06:55:05.947230 26461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.947588 26461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:05.947618 26467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.948056 26467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 06:55:05 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:05.950210 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.942117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.3.247.189:41535)
0806 06:55:05.942161 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.942162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.942207 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:55:05.942276 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.942277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.942284 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.942324 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.942825 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.943238 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.944173 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.944449 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.944504 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.947059 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.947060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.947060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.947067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.948777 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.949197 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.949198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.949199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.949249 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.949451 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.949452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.949452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.949584 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.949728 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.949973 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.950042 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:55:05.949409 26478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:55:05.950739 26478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:55:05.951987 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:55:05.949656 26393 master_runner.cc:186] Error getting master registration for 127.3.247.190:36369: OK, instance_id { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" instance_seqno: 1785999305603720 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:55:05.953454 26393 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:55:05.953315 26285 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44858
I20260806 06:55:05.955910 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.945719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.3.247.188:43409)
0806 06:55:05.945878 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.945880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.945886 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:55:05.945909 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.945910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.945916 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.945973 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.946209 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.946742 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.947593 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.947896 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.947976 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.950425 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.950426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.950426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.950431 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.954925 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.955286 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.955286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.955287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.955330 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.955503 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.955503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.955503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.955603 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.955649 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.955788 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.955843 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260806 06:55:05.955988 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.945671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45682)
0806 06:55:05.945715 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:05.945725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:05.945759 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:05.945978 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:05.945978 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:05.946104 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:05.946134 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.946135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.946807 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.946808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.947498 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:05.947501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.947976 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.947977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:05.948034 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.948919 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:05.948924 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:05.950346 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:05.954994 (+ 4648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.954995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.954995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.955211 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.955385 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:05.955385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:05.955386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:05.955447 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:05.955653 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:05.955654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:05.955700 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 06:55:05.955774 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:55:05.955866 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:55:05.957640 26311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: Bootstrap starting.
I20260806 06:55:05.958391 26311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:05.958698 26311 log.cc:824] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:05.959523 26311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: No bootstrap required, opened a new log
I20260806 06:55:05.960844 26311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.960985 26311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e7b89ab4b14bc1917a9fa63cf2175c, State: Initialized, Role: FOLLOWER
I20260806 06:55:05.961059 26311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.961349 26311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:05.961341 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.961411 26285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:55:05.961536 26285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:05.962423 26285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.962680 26285 leader_election.cc:288] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [CANDIDATE]: Term 1 election: Requested vote from peers b5373a15e1aa40e48e6e7a666b7ed39a (127.3.247.189:41535), 1e929d679eb64a60ac8481eed3366c2e (127.3.247.188:43409)
I20260806 06:55:05.962946 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:05.963208 26435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e929d679eb64a60ac8481eed3366c2e"
I20260806 06:55:05.963342 26435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [term 0 FOLLOWER]: Advancing to term 1
W20260806 06:55:05.963802 26492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:55:05.963898 26492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:55:05.964298 26435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31e7b89ab4b14bc1917a9fa63cf2175c in term 1.
I20260806 06:55:05.964895 26261 leader_election.cc:302] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e929d679eb64a60ac8481eed3366c2e, 31e7b89ab4b14bc1917a9fa63cf2175c; no voters:
I20260806 06:55:05.965207 26481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:05.965476 26481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [term 1 LEADER]: Becoming Leader. State: Replica: 31e7b89ab4b14bc1917a9fa63cf2175c, State: Running, Role: LEADER
I20260806 06:55:05.965620 26481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:05.966338 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e7b89ab4b14bc1917a9fa63cf2175c. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.966471 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:05.965958 26243 master_runner.cc:186] Error getting master registration for 127.3.247.189:41535: OK, instance_id { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" instance_seqno: 1785999305764866 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:55:05.968245 26243 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:55:05.979001 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.970246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.3.247.188:43409)
0806 06:55:05.970285 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.970287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.970480 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:55:05.970576 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.970577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.970587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.970618 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.971424 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.972183 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.973222 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.973512 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.973553 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.975342 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.975343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.975344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.975351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.977384 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.978068 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.978070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.978070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.978134 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.978344 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.978345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.978346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.978482 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.978569 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.978794 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.978870 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:55:05.980156 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.970035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.3.247.190:36369)
0806 06:55:05.970091 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.970092 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.970252 (+ 160us) server_negotiation.cc:407] Connection header received
0806 06:55:05.970365 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.970366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.970405 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.970461 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.971683 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.972335 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.973686 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.973979 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.974040 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.976858 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.976859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.976860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.976868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.978569 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.979116 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.979118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.979118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.979199 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.979613 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.979614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.979615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.979746 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.979850 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.979978 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.980049 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:55:05.981698 26500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:05.981984 26500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:55:05.984033 26435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [term 1 FOLLOWER]: Refusing update from remote peer 31e7b89ab4b14bc1917a9fa63cf2175c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:55:05.984381 26498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:55:05.985822 26467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e7b89ab4b14bc1917a9fa63cf2175c. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.985918 26467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:05.986945 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:05.970577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.3.247.189:41535)
0806 06:55:05.970630 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:05.970632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:05.977941 (+ 7309us) server_negotiation.cc:407] Connection header received
0806 06:55:05.978046 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:05.978048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:05.978059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:05.978090 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:05.978730 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.979475 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.980998 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:05.981317 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:05.981393 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:05.983004 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:05.983005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:05.983005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:05.983013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:05.984965 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.985396 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.985397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.985397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.985470 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:05.985750 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:05.985751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:05.985751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:05.985922 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:05.986427 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:05.986727 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 06:55:05.986820 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:55:05.994913 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.995036 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:05.995062 26467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:05.995133 26467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:05.995187 26504 mvcc.cc:205] Tried to move back new op lower bound from 7315453157307244544 to 7315453157236805632. Current Snapshot: MvccSnapshot[applied={T|T < 7315453157307244544}]
I20260806 06:55:05.995584 26505 mvcc.cc:205] Tried to move back new op lower bound from 7315453157307244544 to 7315453157236805632. Current Snapshot: MvccSnapshot[applied={T|T < 7315453157307244544}]
I20260806 06:55:05.996315 26500 catalog_manager.cc:1347] Generated new cluster ID: a69a8f25458741a3896d55de04550073
I20260806 06:55:05.996362 26500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:55:06.000257 26386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } attempt: 3
Aug 06 06:55:06 dist-test-slave-0kls krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:06.009137 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:06.000554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:43409 (local address 127.0.0.1:45698)
0806 06:55:06.000654 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:06.000665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:06.000707 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:06.000863 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:06.000864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:06.001052 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:06.001103 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:06.001105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:06.002085 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:06.002086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:06.002897 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:06.002899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:06.003389 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:06.003390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:06.003444 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:06.003946 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:06.003951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:06.005377 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:06.008101 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:06.008102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:06.008103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:06.008309 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:06.008502 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:06.008503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:06.008503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:06.008557 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:06.008817 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:06.008818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:06.008872 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:55:06.008934 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:55:06.009030 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 06:55:06.009487 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:06.000675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.3.247.188:43409)
0806 06:55:06.000715 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.000717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.000730 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:55:06.000754 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.000754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.000764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.000840 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.001200 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:06.001994 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:06.002989 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:06.003286 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:06.003340 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:06.005466 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:06.005467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:06.005467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:06.005474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:06.008010 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:06.008385 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:06.008386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:06.008386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:06.008438 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:06.008617 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:06.008618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:06.008618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:06.008737 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:06.008793 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:06.009314 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 06:55:06.009397 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:55:06.011008 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: Bootstrap starting.
I20260806 06:55:06.011713 26386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:06.011843 26500 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:06.012040 26386 log.cc:824] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:06.012631 26500 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:06.013150 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: No bootstrap required, opened a new log
I20260806 06:55:06.016422 26386 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:06.016723 26386 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5373a15e1aa40e48e6e7a666b7ed39a, State: Initialized, Role: FOLLOWER
I20260806 06:55:06.016923 26386 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } }
I20260806 06:55:06.017544 26359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:06.018445 26359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [term 1 FOLLOWER]: Refusing update from remote peer 31e7b89ab4b14bc1917a9fa63cf2175c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:55:06.018708 26481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:55:06.017565 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a"
I20260806 06:55:06.019698 26360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31e7b89ab4b14bc1917a9fa63cf2175c in term 1.
I20260806 06:55:06.017472 26386 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:06.022565 26500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c: Generated new TSK 0
I20260806 06:55:06.022862 26500 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:55:06.023505 26525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:55:06.023603 26525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:55:06.023824 26514 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:06.023970 26514 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:06.024082 26514 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e7b89ab4b14bc1917a9fa63cf2175c. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:06.024190 26514 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:06.024279 26514 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:06.024386 26514 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:06.025563 26512 mvcc.cc:205] Tried to move back new op lower bound from 7315453157418844160 to 7315453157236805632. Current Snapshot: MvccSnapshot[applied={T|T < 7315453157307244544}]
I20260806 06:55:06.025907 26498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e7b89ab4b14bc1917a9fa63cf2175c" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 36369 } } peers { permanent_uuid: "b5373a15e1aa40e48e6e7a666b7ed39a" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41535 } } peers { permanent_uuid: "1e929d679eb64a60ac8481eed3366c2e" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 43409 } } }
I20260806 06:55:06.026013 26498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e7b89ab4b14bc1917a9fa63cf2175c [sys.catalog]: This master's current role is: LEADER
W20260806 06:55:06.047534 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.045925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.3.247.189:41535)
0806 06:55:06.045971 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.045973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.046873 (+ 900us) server_negotiation.cc:407] Connection header received
0806 06:55:06.046979 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.046980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.046991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.047021 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.047464 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33166: BlockingRecv error: recv got EOF from 127.0.0.1:33166 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:55:06.048323 26314 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.046293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.3.247.190:36369)
0806 06:55:06.046331 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.046333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.047815 (+ 1482us) server_negotiation.cc:407] Connection header received
0806 06:55:06.047913 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.047915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.047926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.047955 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.048255 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44886: BlockingRecv error: recv got EOF from 127.0.0.1:44886 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:55:06.049072 26465 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.046588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45706 (local address 127.3.247.188:43409)
0806 06:55:06.046626 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.046628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.048548 (+ 1920us) server_negotiation.cc:407] Connection header received
0806 06:55:06.048643 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.048645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.048655 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.048684 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.048999 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45706: BlockingRecv error: recv got EOF from 127.0.0.1:45706 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:55:06.051038 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.050340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.3.247.189:41535)
0806 06:55:06.050384 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.050386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.050562 (+ 176us) server_negotiation.cc:407] Connection header received
0806 06:55:06.050664 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.050665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.050682 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.050711 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.050989 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33178: BlockingRecv error: recv got EOF from 127.0.0.1:33178 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 06:55:06.054051 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.053227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.3.247.189:41535)
0806 06:55:06.053258 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.053260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.053495 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:55:06.053583 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.053584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.053593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.053620 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.053988 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33194: BlockingRecv error: recv got EOF from 127.0.0.1:33194 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:55:06.057739 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.057043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.3.247.189:41535)
0806 06:55:06.057074 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.057075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.057281 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:55:06.057370 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.057384 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.057394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.057422 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.057697 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33200: BlockingRecv error: recv got EOF from 127.0.0.1:33200 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:55:06.063616 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.062799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.3.247.189:41535)
0806 06:55:06.062846 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.062847 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.063120 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:55:06.063219 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.063220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.063230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.063258 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.063572 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33204: BlockingRecv error: recv got EOF from 127.0.0.1:33204 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 06:55:06.073915 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.072965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.3.247.189:41535)
0806 06:55:06.073006 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.073008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.073264 (+ 256us) server_negotiation.cc:407] Connection header received
0806 06:55:06.073366 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.073367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.073391 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.073422 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.073772 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33208: BlockingRecv error: recv got EOF from 127.0.0.1:33208 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:55:06.092273 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.091235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.3.247.189:41535)
0806 06:55:06.091276 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.091278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.091618 (+ 340us) server_negotiation.cc:407] Connection header received
0806 06:55:06.091719 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.091720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.091731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.091761 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.092212 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33220: BlockingRecv error: recv got EOF from 127.0.0.1:33220 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:55:06.126642 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.125618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.3.247.189:41535)
0806 06:55:06.125662 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.125664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.126031 (+ 367us) server_negotiation.cc:407] Connection header received
0806 06:55:06.126139 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.126140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.126152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.126182 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.126581 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33228: BlockingRecv error: recv got EOF from 127.0.0.1:33228 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:55:06.192942 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.192216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.3.247.189:41535)
0806 06:55:06.192288 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.192290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.192415 (+ 125us) server_negotiation.cc:407] Connection header received
0806 06:55:06.192465 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.192466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.192479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.192544 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.192863 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33232: BlockingRecv error: recv got EOF from 127.0.0.1:33232 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 06:55:06.322934 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.322104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.3.247.189:41535)
0806 06:55:06.322172 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.322174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.322314 (+ 140us) server_negotiation.cc:407] Connection header received
0806 06:55:06.322415 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.322416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.322429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.322499 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.322860 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33234: BlockingRecv error: recv got EOF from 127.0.0.1:33234 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 06:55:06.580816 26390 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:06.580188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.3.247.189:41535)
0806 06:55:06.580254 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:06.580256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:06.580286 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:55:06.580311 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:06.580312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:06.580322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:06.580375 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:06.580728 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33248: BlockingRecv error: recv got EOF from 127.0.0.1:33248 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:55:06.951344 26478 catalog_manager.cc:1259] Loaded cluster ID: a69a8f25458741a3896d55de04550073
I20260806 06:55:06.951426 26478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: loading cluster ID for follower catalog manager: success
I20260806 06:55:06.952246 26478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: acquiring CA information for follower catalog manager: success
I20260806 06:55:06.952507 26478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e929d679eb64a60ac8481eed3366c2e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:55:07.024112 26525 catalog_manager.cc:1259] Loaded cluster ID: a69a8f25458741a3896d55de04550073
I20260806 06:55:07.024189 26525 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: loading cluster ID for follower catalog manager: success
I20260806 06:55:07.025066 26525 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: acquiring CA information for follower catalog manager: success
I20260806 06:55:07.025403 26525 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5373a15e1aa40e48e6e7a666b7ed39a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:55:07.097095 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:07.094043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.3.247.189:41535)
0806 06:55:07.094181 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:07.094183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:07.094261 (+ 78us) server_negotiation.cc:407] Connection header received
0806 06:55:07.094362 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:07.094363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:07.094439 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:07.094522 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:07.094697 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.095441 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.096270 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.096483 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.096555 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:07.096739 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:07.096856 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:07.096948 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:07.096967 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 06:55:07.096991 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260806 06:55:07.101291 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:07.099063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.3.247.188:43409)
0806 06:55:07.099184 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:07.099186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:07.099195 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:55:07.099248 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:07.099248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:07.099283 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:07.099356 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:07.099463 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.099938 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.100626 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.100808 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.100855 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:07.100977 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:07.101078 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:07.101134 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:07.101169 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:55:07.101187 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260806 06:55:07.105772 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:07.103199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.3.247.188:43409)
0806 06:55:07.103233 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:07.103235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:07.103243 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:55:07.103309 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:07.103309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:07.103318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:07.103365 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:07.103539 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.104024 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.104969 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.105253 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.105323 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:07.105487 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:07.105519 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:07.105574 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:07.105661 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:55:07.105684 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:55:07.106106 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:07.102979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.3.247.189:41535)
0806 06:55:07.103021 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:07.103022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:07.103250 (+ 228us) server_negotiation.cc:407] Connection header received
0806 06:55:07.103365 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:07.103366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:07.103378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:07.103451 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:07.103597 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.104348 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.105307 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.105580 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.105636 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:07.105829 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:07.105865 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:07.105919 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:07.106016 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:55:07.106038 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:55:07.106376 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:07.102961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.3.247.190:36369)
0806 06:55:07.103059 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:07.103061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:07.103122 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:55:07.103249 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:07.103250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:07.103304 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:07.103380 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:07.103549 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.104315 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.105350 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:07.105688 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:07.105749 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:07.105989 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:07.106112 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:07.106168 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:07.106268 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:55:07.106295 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":43,"threads_started":1}
I20260806 06:55:07.107932 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26243
I20260806 06:55:07.113574 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26318
I20260806 06:55:07.118499 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26393
2026-08-06T06:55:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2782 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:55:07 dist-test-slave-0kls krb5kdc[26610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:55:07 dist-test-slave-0kls krb5kdc[26610](info): set up 2 sockets
Aug 06 06:55:07 dist-test-slave-0kls krb5kdc[26610](info): commencing operation
krb5kdc: starting...
W20260806 06:55:08.154227 4062 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/test-user.keytab.
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.247.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
Entry for principal kudu/127.3.247.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab.
I20260806 06:55:08.272600 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33273
--webserver_interface=127.3.247.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:55:08.353336 26623 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:08.353554 26623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:08.353571 26623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:08.353626 26623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:08.354854 26623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:08.354897 26623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:08.354911 26623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:08.354955 26623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:08.354986 26623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:08.356520 26623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.190.keytab
--principal=kudu/127.3.247.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.190:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.247.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:08.356788 26623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:08.357035 26623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:08.361744 26631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:08.361819 26629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:08.361756 26628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:08.362079 26623 server_base.cc:1034] running on GCE node
I20260806 06:55:08.362323 26623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:08.362645 26623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:08.363847 26623 hybrid_clock.cc:648] HybridClock initialized: now 1785999308363813 us; error 51 us; skew 500 ppm
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:08.366815 26623 init.cc:377] Logged in from keytab as kudu/127.3.247.190@KRBTEST.COM (short username kudu)
I20260806 06:55:08.367358 26623 webserver.cc:460] Webserver started at http://127.3.247.190:40413/ using document root <none> and password file <none>
I20260806 06:55:08.367635 26623 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:08.367717 26623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:08.367895 26623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:08.369151 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data/instance:
uuid: "d8269217d8f940faa19b67ade6a44c41"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "e4dda6fbb7179131f635ccafe280b08e"
server_key_iv: "7be44878f51e52aa5e1fdabd6bdd802d"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.369578 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance:
uuid: "d8269217d8f940faa19b67ade6a44c41"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "e4dda6fbb7179131f635ccafe280b08e"
server_key_iv: "7be44878f51e52aa5e1fdabd6bdd802d"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.371163 26623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:55:08.372035 26638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:08.372228 26623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:55:08.372372 26623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal
uuid: "d8269217d8f940faa19b67ade6a44c41"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "e4dda6fbb7179131f635ccafe280b08e"
server_key_iv: "7be44878f51e52aa5e1fdabd6bdd802d"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.372466 26623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:08.417479 26623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:08.420367 26623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:08.420570 26623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:08.425408 26623 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.190:33273
I20260806 06:55:08.425444 26690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.190:33273 every 8 connection(s)
I20260806 06:55:08.425881 26623 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/data/info.pb
I20260806 06:55:08.426970 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:08.428043 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26623
I20260806 06:55:08.428143 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-0/wal/instance
I20260806 06:55:08.428368 4062 external_mini_cluster.cc:1428] Setting key cef78cd19d3dbb1bdc1fe685c8aa9aa4
I20260806 06:55:08.429204 26691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.247.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.189@KRBTEST.COM" created.
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
I20260806 06:55:08.443056 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.430205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.3.247.190:33273)
0806 06:55:08.430402 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.430409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.430538 (+ 129us) server_negotiation.cc:407] Connection header received
0806 06:55:08.431191 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.431194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.431223 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.431337 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.431967 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.432866 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.433960 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.434324 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.434478 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.439153 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.439161 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.439161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.439169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.441368 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.441820 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.441823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.441823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.441908 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.442137 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.442138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.442139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.442382 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.442480 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.442664 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.442803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260806 06:55:08.443059 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.430136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33273 (local address 127.0.0.1:47506)
0806 06:55:08.430345 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.430399 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.430982 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.431363 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.431366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.431680 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.431851 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.431855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.433000 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.433001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.433842 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.433848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.434462 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.434463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.434543 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.435213 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.435222 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.439058 (+ 3836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.441470 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.441477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.441478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.441675 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.441983 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.441984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.441984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.442069 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.442457 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.442462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.442525 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.442633 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.442873 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":94,"thread_start_us":92,"threads_started":1}
I20260806 06:55:08.444193 26691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.444646 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.444472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42644)
0806 06:55:08.444541 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.444609 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 06:55:08.444846 26641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
I20260806 06:55:08.445323 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.445233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42648)
0806 06:55:08.445281 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.445306 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:55:08.445451 26691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41851: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
Entry for principal kudu/127.3.247.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab.
I20260806 06:55:08.457247 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:41851
--webserver_interface=127.3.247.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:55:08.494686 26691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } attempt: 1
I20260806 06:55:08.498145 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.497984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42656)
0806 06:55:08.498056 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.498103 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:55:08.498591 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.498496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42668)
0806 06:55:08.498542 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.498570 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:55:08.498744 26691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41851: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
W20260806 06:55:08.544062 26698 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:08.544265 26698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:08.544282 26698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:08.544302 26698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:08.545457 26698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:08.545495 26698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:08.545509 26698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:08.545522 26698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:08.545534 26698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:08.546990 26698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.189.keytab
--principal=kudu/127.3.247.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.189:41851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.247.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:08.547179 26698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:08.547343 26698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:08.552114 26706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:08.552214 26698 server_base.cc:1034] running on GCE node
W20260806 06:55:08.552172 26704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:08.552114 26703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:08.552556 26698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:08.552835 26698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:08.553992 26698 hybrid_clock.cc:648] HybridClock initialized: now 1785999308553984 us; error 38 us; skew 500 ppm
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:08.556324 26698 init.cc:377] Logged in from keytab as kudu/127.3.247.189@KRBTEST.COM (short username kudu)
I20260806 06:55:08.556772 26698 webserver.cc:460] Webserver started at http://127.3.247.189:43709/ using document root <none> and password file <none>
I20260806 06:55:08.557003 26698 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:08.557070 26698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:08.557215 26698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:08.558287 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data/instance:
uuid: "79d04c156d2948a594a5472b9525ecb6"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "c89600c411bf686bb4ad0773e54a01eb"
server_key_iv: "24b19cad5774c1954512971d2e34ddda"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.558657 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal/instance:
uuid: "79d04c156d2948a594a5472b9525ecb6"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "c89600c411bf686bb4ad0773e54a01eb"
server_key_iv: "24b19cad5774c1954512971d2e34ddda"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.559913 26698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:55:08.560590 26713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:08.560750 26698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:55:08.560847 26698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal
uuid: "79d04c156d2948a594a5472b9525ecb6"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "c89600c411bf686bb4ad0773e54a01eb"
server_key_iv: "24b19cad5774c1954512971d2e34ddda"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.560925 26698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:08.566876 26691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } attempt: 2
I20260806 06:55:08.567255 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.567143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42672)
0806 06:55:08.567196 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.567229 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:55:08.567628 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.567511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42688)
0806 06:55:08.567557 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.567589 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:55:08.567721 26691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.189:41851: Network error: Client connection negotiation failed: client connection to 127.3.247.189:41851: connect: Connection refused (error 111)
I20260806 06:55:08.571475 26698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:08.573300 26698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:08.573462 26698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:08.577558 26698 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.189:41851
I20260806 06:55:08.577594 26765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.189:41851 every 8 connection(s)
I20260806 06:55:08.578042 26698 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/data/info.pb
I20260806 06:55:08.579005 26766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:08.580507 26766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.582302 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26698
I20260806 06:55:08.582417 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-1/wal/instance
I20260806 06:55:08.582641 4062 external_mini_cluster.cc:1428] Setting key e2bc2aee3b9542419e872d59cf602bc1
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.247.188@KRBTEST.COM; defaulting to no policy
I20260806 06:55:08.590167 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.581198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.3.247.190:33273)
0806 06:55:08.581260 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.581262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.581463 (+ 201us) server_negotiation.cc:407] Connection header received
0806 06:55:08.581979 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.581980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.581991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.582054 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.582936 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.583441 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.584627 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.584850 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.584940 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.587201 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.587203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.587204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.587212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.588686 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.589128 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.589129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.589129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.589192 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.589441 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.589442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.589442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.589559 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.589642 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.590028 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.590099 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:55:08.590380 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.581176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33273 (local address 127.0.0.1:47510)
0806 06:55:08.581329 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.581372 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.581896 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.582139 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.582144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.582535 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.582843 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.582851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.583538 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.583540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.584528 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.584531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.584963 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.584964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.585092 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.585639 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.585648 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.587102 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.588798 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.588805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.588808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.589049 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.589259 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.589260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.589260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.589370 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.589662 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.589670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.589876 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.590006 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.590151 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260806 06:55:08.591009 26766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.247.188@KRBTEST.COM" created.
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:08.598822 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.591269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42696)
0806 06:55:08.591319 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.591332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.591372 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.591639 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.591640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.591779 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.591823 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.591825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.592892 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.592893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.593393 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.593394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.593929 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.593930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.593994 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.594419 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.594424 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.595819 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.597882 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.597883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.597883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.598036 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.598256 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.598256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.598257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.598305 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.598616 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.598617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.598650 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.598695 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.598760 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:55:08.598937 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.591334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.3.247.189:41851)
0806 06:55:08.591513 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.591517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.591532 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:55:08.591548 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.591550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.591560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.591640 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.591900 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.592802 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.593489 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.593848 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.593933 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.595899 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.595907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.595907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.595913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.597777 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.598116 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.598119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.598120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.598188 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.598360 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.598361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.598361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.598548 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.598635 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.598743 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.598826 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260806 06:55:08.599535 26766 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.599867 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.599742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42304)
0806 06:55:08.599796 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.599844 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:55:08.599998 26714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
I20260806 06:55:08.600425 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.600309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42318)
0806 06:55:08.600359 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.600389 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:55:08.600528 26766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:40735: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
Entry for principal kudu/127.3.247.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab.
I20260806 06:55:08.606532 4062 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktauUQm/build/release/bin/kudu
/tmp/dist-test-tasktauUQm/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:40735
--webserver_interface=127.3.247.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.247.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:55:08.649729 26766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } attempt: 1
I20260806 06:55:08.650285 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.650146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42320)
0806 06:55:08.650210 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.650253 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:55:08.650648 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.650545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42334)
0806 06:55:08.650597 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.650623 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 06:55:08.650740 26766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:40735: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
I20260806 06:55:08.658859 26691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } attempt: 3
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:08.666374 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.659228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.3.247.189:41851)
0806 06:55:08.659272 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.659274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.659390 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:55:08.659419 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.659420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.659431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.659477 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.659856 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.660365 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.661404 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.661630 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.661688 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.663859 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.663860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.663860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.663867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.665244 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.665592 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.665593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.665593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.665652 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.665857 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.665858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.665858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.665970 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.666042 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.666197 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.666275 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:55:08.666384 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.659201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42704)
0806 06:55:08.659258 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.659270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.659335 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.659503 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.659504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.659686 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.659776 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.659777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.660443 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.660444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.661299 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.661303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.661708 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.661709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.661809 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.662241 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.662246 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.663758 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.665329 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.665330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.665330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.665509 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.665718 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.665718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.665719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.665776 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.666045 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.666046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.666091 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.666173 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.666275 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:55:08.666812 26691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.667166 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.667060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42338)
0806 06:55:08.667113 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.667142 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:55:08.667474 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.667420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42352)
0806 06:55:08.667441 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.667460 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 06:55:08.667572 26691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:40735: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
W20260806 06:55:08.687954 26773 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:55:08.688154 26773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:55:08.688170 26773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:55:08.688189 26773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:55:08.689199 26773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:55:08.689229 26773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:55:08.689241 26773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:55:08.689252 26773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:55:08.689263 26773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:55:08.690626 26773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.247.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.247.190:33273,127.3.247.189:41851,127.3.247.188:40735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/krb5kdc/kudu_127.3.247.188.keytab
--principal=kudu/127.3.247.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.247.188:40735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.247.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:55:08.690812 26773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:55:08.690969 26773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:55:08.695657 26779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:08.695652 26781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:55:08.695652 26778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:55:08.695995 26773 server_base.cc:1034] running on GCE node
I20260806 06:55:08.696202 26773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:55:08.696455 26773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:55:08.697593 26773 hybrid_clock.cc:648] HybridClock initialized: now 1785999308697582 us; error 30 us; skew 500 ppm
I20260806 06:55:08.698701 26691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } attempt: 1
I20260806 06:55:08.699102 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.698978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42354)
0806 06:55:08.699039 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.699074 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:55:08.699409 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.699308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42358)
0806 06:55:08.699363 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.699386 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:55:08.699496 26691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:40735: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:55:08.700244 26773 init.cc:377] Logged in from keytab as kudu/127.3.247.188@KRBTEST.COM (short username kudu)
I20260806 06:55:08.700660 26773 webserver.cc:460] Webserver started at http://127.3.247.188:42505/ using document root <none> and password file <none>
I20260806 06:55:08.700881 26773 fs_manager.cc:359] Metadata directory not provided
I20260806 06:55:08.700927 26773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:55:08.701071 26773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:55:08.702140 26773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data/instance:
uuid: "119a4aa25b11489d98f5080a9ae5e833"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "8382c751e49cd136e1706d8caab0fc2d"
server_key_iv: "808d362d6086e30964938b8362b2836f"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.702494 26773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal/instance:
uuid: "119a4aa25b11489d98f5080a9ae5e833"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "8382c751e49cd136e1706d8caab0fc2d"
server_key_iv: "808d362d6086e30964938b8362b2836f"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.703728 26773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:55:08.704509 26788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:55:08.704676 26773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:55:08.704767 26773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data,/tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal
uuid: "119a4aa25b11489d98f5080a9ae5e833"
format_stamp: "Formatted at 2026-08-06 06:55:08 on dist-test-slave-0kls"
server_key: "8382c751e49cd136e1706d8caab0fc2d"
server_key_iv: "808d362d6086e30964938b8362b2836f"
server_key_version: "encryptionkey@0"
I20260806 06:55:08.704821 26773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:55:08.718000 26773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:55:08.718885 26766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } attempt: 2
I20260806 06:55:08.719287 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.719157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42366)
0806 06:55:08.719216 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.719247 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:55:08.719688 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.719609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42378)
0806 06:55:08.719650 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.719673 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:55:08.719799 26766 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.247.188:40735: Network error: Client connection negotiation failed: client connection to 127.3.247.188:40735: connect: Connection refused (error 111)
I20260806 06:55:08.720553 26773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:55:08.720714 26773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:55:08.724596 26773 rpc_server.cc:307] RPC server started. Bound to: 127.3.247.188:40735
I20260806 06:55:08.724647 26840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.247.188:40735 every 8 connection(s)
I20260806 06:55:08.724996 26773 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/data/info.pb
I20260806 06:55:08.726052 26841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:55:08.727531 26841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.730693 4062 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktauUQm/build/release/bin/kudu as pid 26773
I20260806 06:55:08.730824 4062 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktauUQm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785999040366271-4062-0/minicluster-data/master-2/wal/instance
I20260806 06:55:08.731070 4062 external_mini_cluster.cc:1428] Setting key a9a8ed7bceb6fb1ccb5a47a6809ad607
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:08.736663 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.728169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.3.247.190:33273)
0806 06:55:08.728227 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.728229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.728422 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:55:08.728853 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.728854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.728863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.728925 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.729508 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.730044 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.731082 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.731346 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.731425 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.733685 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.733687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.733687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.733694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.735229 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.735638 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.735639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.735640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.735721 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.735979 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.735980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.735980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.736115 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.736185 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.736476 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.736562 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:55:08.736711 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.728118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.190:33273 (local address 127.0.0.1:47518)
0806 06:55:08.728276 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.728317 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.728777 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.728997 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.729001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.729228 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.729439 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.729443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.730116 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.730117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.731002 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.731005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.731457 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.731459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.731616 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.732112 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.732122 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.733587 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.735333 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.735339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.735341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.735560 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.735797 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.735798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.735798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.735907 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.736180 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.736184 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.736289 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.736433 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.736560 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260806 06:55:08.737553 26841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:55:08.740015 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.731450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.3.247.188:40735)
0806 06:55:08.731588 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.731592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.731607 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:55:08.731663 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.731665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.731680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.731739 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.732531 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.733145 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.733976 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.734273 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.734361 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.736978 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.736984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.736984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.736990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.738862 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.739193 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.739196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.739196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.739249 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.739459 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.739460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.739461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.739600 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.739733 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.739836 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.739910 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:08.744290 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.737862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.189:41851 (local address 127.0.0.1:42718)
0806 06:55:08.737901 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.737910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.737944 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.738078 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.738079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.738270 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.738303 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.738304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.738994 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.738996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.739479 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.739481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.739866 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.739867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.739927 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.740430 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.740435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.741693 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.743273 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.743274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.743274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.743503 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.743719 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.743719 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.743719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.743788 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.744086 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.744086 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.744119 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.744170 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.744234 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:55:08.744398 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.737914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.3.247.189:41851)
0806 06:55:08.737964 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.737966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.737976 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:55:08.738001 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.738002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.738011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.738072 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.738435 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.738866 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.739580 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.739795 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.739870 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.741776 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.741777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.741778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.741806 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.743201 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.743576 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.743577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.743578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.743652 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.743872 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.743873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.743874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.744010 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.744092 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.744199 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.744278 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:55:08.744728 26841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.190@KRBTEST.COM
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.188@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:08.748741 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.740778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.3.247.190:33273)
0806 06:55:08.740811 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.740813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.740821 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:55:08.740893 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.740894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.740901 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.740944 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.741371 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.741933 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.742933 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.743105 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.743153 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.745712 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.745714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.745714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.745720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.747242 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.747875 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.747876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.747877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.747934 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.748072 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.748072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.748073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.748191 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.748337 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.748561 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.748638 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:55:08.751066 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.744975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42408)
0806 06:55:08.745020 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.745030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.745063 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.745221 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.745221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.745361 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.745393 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.745394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.746056 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.746057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.746509 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.746510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.746889 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.746890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.746938 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.747293 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.747297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.748428 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.750094 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.750096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.750096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.750300 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.750505 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.750506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.750506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.750554 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.750785 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.750786 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.750814 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.750858 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.750936 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 06:55:08.751121 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.745043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.3.247.188:40735)
0806 06:55:08.745079 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.745081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.745088 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:55:08.745107 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.745108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.745117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.745182 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.745462 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.745974 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.746598 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.746807 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.746890 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.748536 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.748537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.748538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.748543 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.750016 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.750376 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.750377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.750377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.750436 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.750611 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.750612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.750612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.750728 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.750793 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.750879 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.751000 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":21}
I20260806 06:55:08.752467 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: Bootstrap starting.
I20260806 06:55:08.753263 26841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:08.753551 26841 log.cc:824] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: Log is configured to *not* fsync() on all Append() calls
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.247.189@KRBTEST.COM
I20260806 06:55:08.754364 26841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: No bootstrap required, opened a new log
I20260806 06:55:08.756831 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.749473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.3.247.189:41851)
0806 06:55:08.749517 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.749518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.749545 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:55:08.749616 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.749617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.749626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.749668 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.750253 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.750935 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.751813 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.752050 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.752133 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.754218 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.754219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.754220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.754226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.755769 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.756088 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.756089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.756089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.756149 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.756333 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.756334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.756334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.756462 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.756627 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.756668 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.756739 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:55:08.756614 26841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.756878 26841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119a4aa25b11489d98f5080a9ae5e833, State: Initialized, Role: FOLLOWER
I20260806 06:55:08.756964 26841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.757201 26847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.757261 26841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:08.757303 26847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.758226 4062 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 06:55:08.759235 26858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:55:08.759285 26858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:55:08.759187 26665 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d8269217d8f940faa19b67ade6a44c41"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47526
I20260806 06:55:08.759836 26773 master_runner.cc:186] Error getting master registration for 127.3.247.190:33273: OK, instance_id { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" instance_seqno: 1785999308424152 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:55:08.762701 26773 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:55:08.774626 26691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } attempt: 2
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.190@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:08.782662 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.775063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.3.247.188:40735)
0806 06:55:08.775110 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.775112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.775149 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:55:08.775235 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.775236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.775248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.775315 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.775719 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.776290 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.777269 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.777601 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.777671 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.779921 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.779922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.779923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.779931 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.781352 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.781743 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.781744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.781745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.781840 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.782034 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.782035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.782036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.782240 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.782299 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.782499 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.782569 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:55:08.782662 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.774979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42410)
0806 06:55:08.775049 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.775100 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.775160 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.775365 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.775366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.775572 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.775628 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.775630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.776366 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.776367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.777166 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.777168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.777678 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.777679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.777740 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.778300 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.778306 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.779813 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.781460 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.781461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.781462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.781681 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.781921 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.781922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.781922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.781983 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.782323 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.782325 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.782399 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.782464 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.782560 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:55:08.784137 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: Bootstrap starting.
I20260806 06:55:08.784683 26691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:08.784916 26691 log.cc:824] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:08.785652 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: No bootstrap required, opened a new log
I20260806 06:55:08.786652 26691 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.786758 26691 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8269217d8f940faa19b67ade6a44c41, State: Initialized, Role: FOLLOWER
I20260806 06:55:08.786811 26691 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.787034 26861 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.787101 26691 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:08.787106 26861 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.787225 26665 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:55:08.787344 26665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:08.788175 26665 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
W20260806 06:55:08.788975 26872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:55:08.789033 26872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:55:08.789762 26623 master_runner.cc:186] Error getting master registration for 127.3.247.189:41851: OK, instance_id { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" instance_seqno: 1785999308576408 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:55:08.792183 26623 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:55:08.792359 26665 leader_election.cc:288] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [CANDIDATE]: Term 1 election: Requested vote from peers 79d04c156d2948a594a5472b9525ecb6 (127.3.247.189:41851), 119a4aa25b11489d98f5080a9ae5e833 (127.3.247.188:40735)
I20260806 06:55:08.792687 26815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8269217d8f940faa19b67ade6a44c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "119a4aa25b11489d98f5080a9ae5e833"
I20260806 06:55:08.792814 26815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:08.794214 26815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8269217d8f940faa19b67ade6a44c41 in term 1.
I20260806 06:55:08.794449 26639 leader_election.cc:302] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119a4aa25b11489d98f5080a9ae5e833, d8269217d8f940faa19b67ade6a44c41; no voters:
I20260806 06:55:08.794567 26861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:55:08.794703 26861 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [term 1 LEADER]: Becoming Leader. State: Replica: d8269217d8f940faa19b67ade6a44c41, State: Running, Role: LEADER
I20260806 06:55:08.794799 26861 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.795207 26861 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8269217d8f940faa19b67ade6a44c41. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.795307 26861 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:08.795499 26880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:55:08.795688 26880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:55:08.798188 26815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [term 1 FOLLOWER]: Refusing update from remote peer d8269217d8f940faa19b67ade6a44c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:55:08.798660 26861 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:55:08.800112 26847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8269217d8f940faa19b67ade6a44c41. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.800222 26847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.809429 26861 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.809535 26861 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:08.810079 26847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.810173 26847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.810335 26884 mvcc.cc:205] Tried to move back new op lower bound from 7315453168833888256 to 7315453168824233984. Current Snapshot: MvccSnapshot[applied={T|T < 7315453168833888256}]
I20260806 06:55:08.810599 26885 mvcc.cc:205] Tried to move back new op lower bound from 7315453168833888256 to 7315453168824233984. Current Snapshot: MvccSnapshot[applied={T|T < 7315453168833888256}]
I20260806 06:55:08.810941 26766 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } attempt: 3
I20260806 06:55:08.812196 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.794598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.3.247.189:41851)
0806 06:55:08.794643 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.794645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.797589 (+ 2944us) server_negotiation.cc:407] Connection header received
0806 06:55:08.797916 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.797918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.797929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.797959 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.799378 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.800095 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.801501 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.802224 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.802271 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.806778 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.806779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.806779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.806787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.809261 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.810444 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.810445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.810446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.810504 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.810698 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.810699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.810699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.810817 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.810918 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.812014 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.812093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:55:08.812976 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.794296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.3.247.188:40735)
0806 06:55:08.794367 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.794369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.797495 (+ 3126us) server_negotiation.cc:407] Connection header received
0806 06:55:08.797768 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.797769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.797781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.797833 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.801343 (+ 3510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.802070 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.803785 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.804110 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.804148 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.807550 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.807551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.807551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.807558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.809216 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.810974 (+ 1758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.810975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.810976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.811029 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.812446 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.812448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.812448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.812579 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.812640 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.812813 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.812892 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:55:08.813804 26880 catalog_manager.cc:1347] Generated new cluster ID: 447112fc49e546049be406a53084bfc3
I20260806 06:55:08.813848 26880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:55:08 dist-test-slave-0kls krb5kdc[26610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999308, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.247.189@KRBTEST.COM for kudu/127.3.247.188@KRBTEST.COM
I20260806 06:55:08.819690 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.794989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.3.247.190:33273)
0806 06:55:08.795027 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.795029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.801458 (+ 6429us) server_negotiation.cc:407] Connection header received
0806 06:55:08.803763 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.803765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.803784 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.803848 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.805205 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.805922 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.811108 (+ 5186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.811400 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.811440 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.815745 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.815747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.815747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.815756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.817701 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.818505 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.818507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.818507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.818568 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.818943 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.818944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.818944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.819077 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.819160 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.819511 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.819593 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:55:08.819833 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.811153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.247.188:40735 (local address 127.0.0.1:42414)
0806 06:55:08.811224 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:55:08.811233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:55:08.811263 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:55:08.813166 (+ 1903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:55:08.813167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:55:08.813327 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:55:08.813379 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.813381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.814234 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.814235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.815012 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:55:08.815014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.815463 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.815464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:55:08.815523 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.815908 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:55:08.815913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:55:08.817314 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:55:08.818941 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.818942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.818942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.819131 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.819289 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:55:08.819289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:55:08.819289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:55:08.819333 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:55:08.819579 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:55:08.819580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:55:08.819616 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:55:08.819666 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:55:08.819745 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 06:55:08.820149 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.811343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.3.247.188:40735)
0806 06:55:08.813059 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.813062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.813069 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:55:08.813087 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.813088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.813096 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.813160 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.813464 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.814150 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.815091 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.815376 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.815422 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.817403 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:55:08.817404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:55:08.817404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:55:08.817410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:55:08.818862 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.819193 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.819194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.819194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.819236 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:55:08.819407 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:55:08.819408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:55:08.819408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:55:08.819511 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:55:08.819559 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.819980 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.820066 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1702,"thread_start_us":85,"threads_started":1}
I20260806 06:55:08.821512 26766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: Bootstrap starting.
I20260806 06:55:08.822187 26766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: Neither blocks nor log segments found. Creating new log.
I20260806 06:55:08.822463 26766 log.cc:824] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:55:08.823462 26766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: No bootstrap required, opened a new log
I20260806 06:55:08.826339 26880 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:55:08.827049 26880 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:55:08.829562 26766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.829964 26766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d04c156d2948a594a5472b9525ecb6, State: Initialized, Role: FOLLOWER
I20260806 06:55:08.830210 26766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } }
I20260806 06:55:08.831051 26739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:55:08.832047 26739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [term 1 FOLLOWER]: Refusing update from remote peer d8269217d8f940faa19b67ade6a44c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:55:08.832212 26880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41: Generated new TSK 0
I20260806 06:55:08.832366 26880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:55:08.832366 26878 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:55:08.831077 26740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8269217d8f940faa19b67ade6a44c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "79d04c156d2948a594a5472b9525ecb6"
I20260806 06:55:08.833624 26740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8269217d8f940faa19b67ade6a44c41 in term 1.
I20260806 06:55:08.833964 26892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.834081 26892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.836764 26895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8269217d8f940faa19b67ade6a44c41. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.836849 26895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.836915 26895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:08.836956 26895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:55:08.830951 26766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:55:08.842859 26893 mvcc.cc:205] Tried to move back new op lower bound from 7315453168971685888 to 7315453168824233984. Current Snapshot: MvccSnapshot[applied={T|T < 7315453168833888256}]
I20260806 06:55:08.846547 26907 catalog_manager.cc:1259] Loaded cluster ID: 447112fc49e546049be406a53084bfc3
I20260806 06:55:08.846653 26907 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: loading cluster ID for follower catalog manager: success
I20260806 06:55:08.847849 26907 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: acquiring CA information for follower catalog manager: success
I20260806 06:55:08.848248 26907 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79d04c156d2948a594a5472b9525ecb6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 06:55:08.851933 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.850566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.3.247.190:33273)
0806 06:55:08.850611 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.850613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.851256 (+ 643us) server_negotiation.cc:407] Connection header received
0806 06:55:08.851364 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.851365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.851376 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.851408 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.851868 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47536: BlockingRecv error: recv got EOF from 127.0.0.1:47536 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:55:08.852648 26770 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.850955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.3.247.189:41851)
0806 06:55:08.850994 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.850995 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.852161 (+ 1166us) server_negotiation.cc:407] Connection header received
0806 06:55:08.852259 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.852260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.852270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.852302 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.852595 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42762: BlockingRecv error: recv got EOF from 127.0.0.1:42762 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:55:08.853732 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.852995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.3.247.188:40735)
0806 06:55:08.853043 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.853044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.853170 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:55:08.853265 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.853265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.853277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.853321 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.853659 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42426: BlockingRecv error: recv got EOF from 127.0.0.1:42426 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:55:08.858702 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.855282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.3.247.189:41851)
0806 06:55:08.855317 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.855319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.855522 (+ 203us) server_negotiation.cc:407] Connection header received
0806 06:55:08.855627 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.855628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.855656 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.855699 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.855931 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.856626 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.857733 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.858053 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.858100 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.858334 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.858449 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.858497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.858614 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.858631 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:55:08.862001 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.861163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.3.247.188:40735)
0806 06:55:08.861202 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.861204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.861391 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:55:08.861503 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.861504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.861524 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.861556 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.861947 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42430: BlockingRecv error: recv got EOF from 127.0.0.1:42430 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:55:08.867739 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.864343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.3.247.189:41851)
0806 06:55:08.864381 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.864383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.864642 (+ 259us) server_negotiation.cc:407] Connection header received
0806 06:55:08.864745 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.864746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.864756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.864812 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.864995 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.865672 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.866956 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.867236 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.867279 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.867483 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.867518 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.867544 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.867644 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.867666 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:55:08.870987 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.870172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.3.247.188:40735)
0806 06:55:08.870213 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.870215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.870404 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:55:08.870508 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.870509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.870523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.870554 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.870934 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42442: BlockingRecv error: recv got EOF from 127.0.0.1:42442 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:55:08.877453 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.874193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.3.247.189:41851)
0806 06:55:08.874228 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.874230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.874469 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:55:08.874571 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.874572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.874581 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.874608 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.874768 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.875553 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.876623 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.876922 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.876963 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.877198 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.877230 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.877257 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.877361 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.877380 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:55:08.880458 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.879691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.3.247.188:40735)
0806 06:55:08.879726 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.879728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.879920 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:55:08.880020 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.880021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.880033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.880065 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.880411 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42448: BlockingRecv error: recv got EOF from 127.0.0.1:42448 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:55:08.888799 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.885617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.3.247.189:41851)
0806 06:55:08.885652 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.885654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.885949 (+ 295us) server_negotiation.cc:407] Connection header received
0806 06:55:08.886059 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.886060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.886069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.886097 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.886267 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.886948 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.887995 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.888266 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.888309 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.888526 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.888561 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.888589 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.888694 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.888712 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:55:08.892671 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.891864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.3.247.188:40735)
0806 06:55:08.891907 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.891909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.892100 (+ 191us) server_negotiation.cc:407] Connection header received
0806 06:55:08.892203 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.892203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.892217 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.892248 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.892614 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42462: BlockingRecv error: recv got EOF from 127.0.0.1:42462 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:55:08.905402 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.901992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.3.247.189:41851)
0806 06:55:08.902035 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.902037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.902318 (+ 281us) server_negotiation.cc:407] Connection header received
0806 06:55:08.902437 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.902438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.902449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.902480 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.902689 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.903450 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.904598 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.904887 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.904934 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.905138 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.905173 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.905201 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.905304 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.905324 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:55:08.908782 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.907953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.3.247.188:40735)
0806 06:55:08.907991 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.907994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.908181 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:55:08.908337 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.908338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.908349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.908378 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.908731 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42468: BlockingRecv error: recv got EOF from 127.0.0.1:42468 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:55:08.929481 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.926139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.3.247.189:41851)
0806 06:55:08.926183 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.926185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.926473 (+ 288us) server_negotiation.cc:407] Connection header received
0806 06:55:08.926580 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.926581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.926591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.926620 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.926807 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.927554 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.928665 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.928957 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.929001 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.929206 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.929242 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.929269 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.929376 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.929396 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:55:08.932732 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.931925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.3.247.188:40735)
0806 06:55:08.931967 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.931969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.932163 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:55:08.932264 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.932265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.932278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.932308 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.932674 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42472: BlockingRecv error: recv got EOF from 127.0.0.1:42472 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:55:08.968497 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:08.966008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.3.247.189:41851)
0806 06:55:08.966056 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.966059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.966215 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:55:08.966288 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.966289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.966300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.966374 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:08.966532 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.967011 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.967780 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:08.967979 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:08.968053 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:08.968195 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:08.968246 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:08.968304 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:08.968382 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 06:55:08.968400 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:55:08.974546 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:08.970331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.3.247.188:40735)
0806 06:55:08.970374 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:08.970376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:08.973969 (+ 3593us) server_negotiation.cc:407] Connection header received
0806 06:55:08.974048 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:08.974049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:08.974067 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:08.974136 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:08.974481 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42478: BlockingRecv error: recv got EOF from 127.0.0.1:42478 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:55:09.042049 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.039757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.3.247.189:41851)
0806 06:55:09.039832 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.039834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.039911 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:55:09.039959 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.039960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.039970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.040039 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.040154 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.040682 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.041512 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.041685 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.041729 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.041860 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.041885 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.041921 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.041972 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.041988 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:55:09.044448 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:09.043731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42486 (local address 127.3.247.188:40735)
0806 06:55:09.043805 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.043807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.043911 (+ 104us) server_negotiation.cc:407] Connection header received
0806 06:55:09.043981 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.043982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.043997 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.044067 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:09.044380 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42486: BlockingRecv error: recv got EOF from 127.0.0.1:42486 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:55:09.176781 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.173835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.3.247.189:41851)
0806 06:55:09.173895 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.173898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.173917 (+ 19us) server_negotiation.cc:407] Connection header received
0806 06:55:09.174062 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.174063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.174077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.174176 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.174398 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.175134 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.176076 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.176259 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.176353 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.176465 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.176500 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.176542 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.176645 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.176668 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:55:09.179085 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:09.178493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.3.247.188:40735)
0806 06:55:09.178536 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.178538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.178549 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:55:09.178613 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.178614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.178625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.178671 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:09.179011 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42490: BlockingRecv error: recv got EOF from 127.0.0.1:42490 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:55:09.387636 27009 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8269217d8f940faa19b67ade6a44c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8269217d8f940faa19b67ade6a44c41" member_type: VOTER last_known_addr { host: "127.3.247.190" port: 33273 } } peers { permanent_uuid: "79d04c156d2948a594a5472b9525ecb6" member_type: VOTER last_known_addr { host: "127.3.247.189" port: 41851 } } peers { permanent_uuid: "119a4aa25b11489d98f5080a9ae5e833" member_type: VOTER last_known_addr { host: "127.3.247.188" port: 40735 } } }
I20260806 06:55:09.387840 27009 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8269217d8f940faa19b67ade6a44c41 [sys.catalog]: This master's current role is: LEADER
I20260806 06:55:09.438894 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.436365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.3.247.189:41851)
0806 06:55:09.436449 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.436451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.436466 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:55:09.436574 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.436575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.436585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.436661 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.436806 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.437330 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.438211 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.438403 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.438483 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.438622 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.438661 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.438719 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.438780 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.438799 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 06:55:09.441371 26890 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:55:09.440640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.3.247.188:40735)
0806 06:55:09.440702 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.440705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.440794 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:55:09.440864 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.440865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.440880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.440946 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:55:09.441276 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42502: BlockingRecv error: recv got EOF from 127.0.0.1:42502 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:55:09.759862 26858 catalog_manager.cc:1259] Loaded cluster ID: 447112fc49e546049be406a53084bfc3
I20260806 06:55:09.759930 26858 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: loading cluster ID for follower catalog manager: success
I20260806 06:55:09.760708 26858 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: acquiring CA information for follower catalog manager: success
I20260806 06:55:09.760957 26858 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 119a4aa25b11489d98f5080a9ae5e833: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:55:09.956885 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.954569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.3.247.189:41851)
0806 06:55:09.954685 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.954687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.954697 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:55:09.954757 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.954757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.954768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.954838 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.954946 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.955443 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.956166 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.956404 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.956469 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.956586 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.956616 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.956720 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.956790 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.956810 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260806 06:55:09.961093 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.958619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.3.247.188:40735)
0806 06:55:09.958723 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.958725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.958732 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:55:09.958771 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.958771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.958818 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.958876 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.958963 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.959525 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.960219 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.960441 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.960500 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.960670 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.960790 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.960872 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.960926 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.960946 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":51,"threads_started":1}
I20260806 06:55:09.965865 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.963040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.3.247.188:40735)
0806 06:55:09.963115 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.963116 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.963127 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:55:09.963161 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.963162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.963174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.963253 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.963480 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.964280 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.965097 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.965329 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.965449 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.965565 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.965598 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.965648 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.965713 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.965734 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:55:09.966337 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.963038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.3.247.189:41851)
0806 06:55:09.963088 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.963091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.963270 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:55:09.963359 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.963360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.963372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.963433 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.963583 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.964310 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.965382 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.965725 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.965830 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.966041 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.966074 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.966104 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.966224 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.966251 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:55:09.966683 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:55:09.963190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.3.247.190:33273)
0806 06:55:09.963407 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 06:55:09.963409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:55:09.963474 (+ 65us) server_negotiation.cc:407] Connection header received
0806 06:55:09.963568 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:55:09.963569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:55:09.963623 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:55:09.963705 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:55:09.963837 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.964627 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.965606 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:55:09.965987 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:55:09.966051 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:55:09.966243 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:55:09.966341 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:55:09.966418 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:55:09.966556 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:55:09.966583 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":52,"threads_started":1}
I20260806 06:55:09.972200 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26623
I20260806 06:55:09.977571 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26698
I20260806 06:55:09.982537 4062 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktauUQm/build/release/bin/kudu with pid 26773
2026-08-06T06:55:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2863 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5646 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269621 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST